run_ingest

dscat.ingest.run_ingest(root, only=None, convert_docs=False)

Build or refresh the catalogue index from data/.

For each configured dataset, discovers its versions, parses their dictionaries, resolves the physical CSVs, and writes the dataset, version, table, feature, and document rows into the index. Synonyms are reloaded and the full-text index is rebuilt at the end. Progress is reported on a single stderr progress bar, advancing as each data CSV is scanned for its row count.

Parameters:

• root (pathlib.Path) – Repository root holding data/ and .catalogue/.

• only (list of str, optional) – Restrict ingestion to these dataset ids; None ingests every configured dataset. Datasets not listed keep their existing rows.

• convert_docs (bool, default False) – After the index is committed, convert every discovered document to markdown (cached under .catalogue/docs/) with the per-format engines, advancing the same progress bar. This is slow, so it is off by default; otherwise conversion happens lazily on first read. A document that fails to convert is reported and skipped.

Returns:

One summary per ingested dataset.

Return type:

list of dscat.ingest.IngestSummary