DatasetConfig

class dscat.config.DatasetConfig(name, display_name, container, layout, version_pattern, dictionary_glob, columns=<factory>, skip_sheets=<factory>, sheet=None, group_by=None, fieldid_row=0, roles=<factory>, file_glob='**/*.csv', strip_version_suffix=False)

One dataset’s adapter configuration, loaded from a JSON file.

Maps a dataset’s data dictionary onto the normalised index. Built-in configs ship in dscat/datasets; a project may override them by name with <root>/datasets/*.json.

name

Dataset id, for example "spark".

Type:

str

display_name

Human-facing label.

Type:

str

container

Subdirectory of data/ to scan for versions.

Type:

str

layout

Layout engine, either "sheet_per_table" or "single_sheet".

Type:

str

version_pattern

Regex with a named group (?P<v>...) that extracts the version from each ship-folder name.

Type:

str

dictionary_glob

Glob, relative to a version directory, that locates the dictionary file.

Type:

str

columns

Canonical field name mapped to header aliases; the first alias that matches a header wins.

Type:

dict of str to list of str

skip_sheets

For sheet_per_table, dictionary sheets to ignore.

Type:

list of str

sheet

For single_sheet, the sheet holding every variable.

Type:

str or None

group_by

For single_sheet, the column that names each variable’s table.

Type:

str or None

fieldid_row

For single_sheet, the 0-based row index of the machine-id header.

Type:

int

roles

For single_sheet, a role mapped to its physical folder or folders under each version directory.

Type:

dict of str to object

file_glob

Glob for the data CSVs (default **/*.csv).

Type:

str

strip_version_suffix

Whether to strip a trailing -<version> from CSV stems before binding them to dictionary tables.

Type:

bool