Parsing the SSC milestone ages#

The developmental-milestone ages drive the “developmental” category that defines the Mixed ASD with developmental delay class, so they have to cross from the SSC into the shared schema cleanly. In SPARK each milestone is a numeric age in months (walked_age_mos and the rest), coded from a dropdown. In the SSC the same milestones are stored as free text, and the entries vary widely: age_walked_alone alone has more than four hundred distinct spellings, from 13 months and 1 yr to a bare 12. The SSC backend turns that free text into months with parse_age_months, in analysis.cohort.schema.

The parsing is the package’s own work. Litman et al. read these milestones from a hand-cleaned background-history file that was never released, and it is the only SSC instrument they pre-clean out of band (every other instrument is harmonised inline in their generate_ssc_data). Without their cleaning recipe, the rules here are ours. They are a documented deviation from the released pipeline, and their fidelity is assessed at the replication stage rather than asserted.

Why a plain numeric coercion is not enough#

The SSC milestone columns are object-typed text, so pandas.to_numeric keeps only the entries that are already bare numbers and turns everything else into a missing value. On the SSC 15.3 proband release that recovers under a tenth of the recorded ages, which collapses the complete-case sample before the model ever runs. Parsing the free text instead recovers the large majority of the recorded values, so the cross-cohort sample is no longer thrown away at this step.

What the parser recognises#

Each entry is lower-cased and lightly cleaned (surrounding punctuation, tildes, commas, and plus signs removed), then matched against a set of forms:

a number with a month unit is read as months: 13 months, 13 mos, 12 mon, 13m. Common abbreviations and typos for “months” are included. A bare number with no unit is read on the scale the SPARK distribution for that milestone makes more likely, so 12 for an early milestone is twelve months and 4 for a late one is four years (see resolving the months-or-years scale).
a number with a year unit is multiplied by twelve, with an optional trailing months part: 1 year, 2 yrs, 1 yr 6 mo.
a number with a week unit is converted from weeks, using 30.4375 days per month (the mean Gregorian month): 6 weeks, 6 wks, 1 week.
a compound entry sums its distinct unit parts: 2 yrs 10 mos, 1 year 2 weeks, 3y;3m.
a half- or quarter-year fraction is read as a decimal first: 3 1/2 yrs, 2 1/2 years, and the half, quarter, and three-quarter fraction glyphs.
a “years old” form is handled, whether written out or abbreviated: 4 years old, 3 y.o., 4 y/o, and age 4.
at birth is zero.
a range, or an “N or M” entry, is read as the midpoint of its endpoints: 12-14, 18 months to 2 years, 7 or 8 months. When the left endpoint is a bare number it borrows the right endpoint’s unit, so 1-2 years reads as one to two years rather than one month to two years.
a bound or inequality has the bound dropped and the stated age taken: <3 mos, >42 mos, before 1 year, under 12 months. HTML-escaped angle brackets (<, >) are decoded first.
an explicit statement that the milestone was never reached (never, not yet, hasn't walked, unable to) is read as the SPARK 888 = not yet code, not as a missing value, so a proband with a severe delay is kept rather than dropped at the complete-case step. The cue has to carry no digit, so a dated note such as 12 mos (lost at 15 mos) is handled as a regression narrative below, not as a never-reached milestone.

Resolving the months-or-years scale#

A free-text age often omits its unit, and a bare number is then ambiguous: 4 is four months for an early milestone such as smiling, but four years for a late one such as bowel training. The parser settles the scale for each milestone against SPARK, where the same milestone is recorded as a clean age in months. For a unit-less number \(x\) it compares how likely \(x\) months and \(12x\) months (the years reading) are under that milestone’s SPARK distribution, estimated as a kernel density in log-age, and takes the more likely; the years reading is considered only when \(12x\) stays within the milestone range.

The result follows each milestone’s own distribution. A small number stays in months where children reach the milestone in months (a child first walks at 13, read as thirteen months), and becomes years where they reach it in years (a child is bowel trained at 4, read as forty-eight months). A written unit is taken as written; only unit-less numbers are resolved this way, including a bare number left after a bound such as under 2.

The reference distribution is the SPARK proband background-history milestones at the release the reference fit uses. The resolver is built per milestone by analysis.cohort.schema.build_milestone_disambiguator() and applied in the SSC backend.

What it leaves missing, and why#

Some entries carry no single age, so the parser returns a missing value and they drop at the complete-case step, exactly as a missing milestone would. Leaving them missing is a choice, not a gap:

text with no number: normal, on time, unsure, within normal limits. These are not ages. (A never or not yet that states the milestone was not reached is the exception above, read as the SPARK not-yet code.)
a calendar date entered in the age field: 03/2003, 12/01, 09/27/93. Reading these as a number of months would be wrong, so they are excluded.
a regression or loss narrative: 12 mos (lost at 15 mos), 18 months then declined till after 2. These describe a trajectory with more than one age. A compound is summed only when its units are distinct and it carries no loss or sequencing cue, so a repeated unit or a word like “lost”, “stopped”, “regressed”, or “then” sends the entry to missing rather than to an arbitrary single value.
a milestone recorded in days, such as 1 day or 2 days. For these milestones a value in days is more likely a stray entry than a real age, so days are not converted.

Choices worth stating#

The parsed ages stay continuous in months. SPARK codes its milestone dropdown in whole months above the first two years, but the features are modelled with a Gaussian density, so a value such as 6 weeks is kept as roughly 1.4 months rather than rounded to the dropdown grid.
A week is converted at 30.4375 days per month, the mean length of a Gregorian month, so weeks and months sit on one continuous scale.
A parsed age above the SPARK over 7 years code is capped at 85 months, the top of the SPARK milestone dropdown, which also discards the occasional mis-parsed outlier.

Result#

On the SSC 15.3 proband release the parser reads about 97 per cent of the non-missing milestone entries, against under a tenth for a plain numeric coercion. The values that remain unread are dominated by the genuine non-ages above: free text with no number, calendar dates, and regression narratives. The ceiling is the data itself, since a share of SSC milestone responses are qualitative rather than point ages, and that is a fidelity limit on the SSC replication regardless of how the text is parsed.

The SSC records one combining-language milestone, mapped to combined_words_age_mos. Two SPARK background-history features have no SSC source: the second combining milestone combined_phrases_age_mos, and the school item repeat_grade. That is a coverage gap in the instrument, separate from parsing, so the SSC backend provides the nine milestones the SSC collects.

The parsing entry point is analysis.cohort.schema.parse_age_months(). For where it sits in the cohort layer, see the cohort interface.