# Subsetting the cohort to the V9 freeze

Litman et al. (2025) fit on a SPARK release this project does not hold. This guide explains how
the `cohort` stage cuts a later release back in time to the probands present at an earlier
freeze, so the model can be refit on an approximation of the data the authors used.

## The release they used

The paper names the release only as "SPARK Phenotype Dataset V9". The released preprocessing
pins the date, reading from a `SPARK_collection_v9_2022-12-12` directory and files such as
`scq_2022-12-12.csv`. So V9 was frozen on 2022-12-12, and that date is what the cut works from.

This project holds the later `2025-03-31` and `2026-03-23` releases. SPARK grows over time, so
`2026-03-23` holds every V9 proband still enrolled plus everyone recruited since. The reference
fit on it recovers the four classes but diverges from the paper in its class proportions, and a
natural question is whether the records added since V9 are the cause. Answering it needs the
cohort cut back to the freeze.

## What the cut recovers, and what it does not

The cut keeps a proband only if it was present at the freeze: enrolled by 2022-12-12 with each of
its four instruments completed by then. This recovers the V9 *roster*, the probands the authors
could have drawn from.

It does not recover the V9 *data*. SPARK revises phenotype between releases, back-filling
corrections and adding responses completed after a proband enrolled, so a kept proband carries
its `2026-03-23` values, not its V9 ones. The cut isolates which probands were present, not which
values were recorded then; matching the values would need the V9 file, which is not held. The
companion investigation reads the cut against a size-matched random subsample, so that "fewer
records" and "different records" can be told apart.

## The two gates

A proband enters only when both gates pass. Both turn on the calendar year, the resolution of the
date fields SPARK exposes.

The roster gate keeps a proband whose `core_descriptive_variables.registration_year` is at or
before 2022.

The completion gate keeps an instrument only if it was completed by the freeze year, read two
ways across the four cohort instruments:

- SCQ, RBS-R, and the background-history forms (child and sibling) carry a completion year,
  `eval_year`, so the gate is `eval_year <= 2022`, exact to the year.
- CBCL 6-18 carries no completion year, only the age at evaluation, so its year is reconstructed
  from the registration anchor,

  $$ \text{cbcl year} = \text{registration year} + (\text{age at evaluation} - \text{age at registration}), $$

  and the proband is kept when that year is below 2023, so a CBCL completed midway through 2022 is
  kept rather than rounded up.

A proband missing any gate field drops out, since the gate cannot then be confirmed. Because the
cohort is reduced to complete cases over all four instruments, a proband survives only when every
instrument cleared the freeze, the "present in V9" condition.

## Running it

The cut is the `--as-of` option on `cohort` and on every stage that builds one:

```
uv run analysis cohort -d spark -v 2026-03-23 --as-of 2022-12-12
```

It enters the run hash, so the subset is cached under its own hash and the full-release reference
artefacts are untouched. Passing `--as-of` to `fit`, `align`, `select`, `stability`, or
`replicate` fits on the subset throughout; for `replicate` the cut applies to the SPARK training
cohort only, since it is a SPARK timing field, and the SSC is projected onto in full.

The size-matched control is `--sample-n`, a random draw of the full release at a fixed
`--sample-seed`:

```
uv run analysis cohort -d spark -v 2026-03-23 --sample-n 5324 --sample-seed 0
```

Set `--sample-n` to the subset's size and the two fits differ only in which probands they hold,
not how many, so a difference between them is compositional rather than a matter of sample size.
The subset size and the results of fitting on it are reported in the investigation on the records
added since V9.