Synonyms#

dscat search expands each query term through a synonym table before it runs the full-text match, so a search finds features that use a different word for the same idea. Searching sleep also matches insomnia and naps; searching iq also matches cognitive, Mullen, and WISC.

How expansion works#

A query is split into terms. Each term becomes an OR-group of the term and its synonyms, and the groups are combined with AND. So sleep problems becomes, roughly:

(sleep* OR insomnia* OR naps*) AND (problems*)

The expanded expression is matched with SQLite FTS5 (BM25 ranking), and results are ordered best first. Expansion improves recall: you find the relevant features without having to guess the exact wording the dataset used.

To search without expansion, pass --raw, which sends your text straight to FTS5 as a MATCH expression (useful for OR, NEAR, prefix *, and column filters).

The synonym file#

Synonyms live in a JSON file: an array of groups, where each group is an array of equivalent terms.

[
  ["sleep", "insomnia", "somnolence", "bedtime", "nap"],
  ["iq", "cognitive", "intelligence", "mullen", "wisc"]
]

A group is bidirectional, so every term expands to every other and the first term is simply a convenient label. Terms are lower-cased and matched against single query tokens, so they apply to words rather than phrases. Keep entries lexical (word stems the full-text tokeniser will match), rather than building an exhaustive ontology.

Built-in and project synonyms#

Two files are read, in order:

  1. The built-in groups that ship with the package, at packages/dscat/src/dscat/synonyms.json.

  2. An optional project file at <repo-root>/synonyms.json, whose groups are appended to the built-ins.

To add your own, create or edit <repo-root>/synonyms.json, add a group as an array of terms, and re-run uv run dscat ingest to load them. The synonym table is rebuilt on each ingest.

A note on spelling#

The datasets ship American spellings in their variable names (for example behavior), so the built-in groups bridge British and American forms where that helps recall. Searching behaviour still matches the behavior variables, because the two sit in the same group.