Tailoring Screening Algorithms for a Large Health System: Autism Screening as a Case Example

When health care systems develop algorithms to implement national screening guidelines, they must choose which information to incorporate (e.g., primary screening, secondary screening, and results from prior visits), whether to follow recommended screening thresholds (i.e., cut scores), and if not, how to set them. Our objective was to analyze electronic medical record (EMR) data to inform these choices.

Retrospective analyses utilized EMR from 33,490 families of children 4 years of age and under enrolled in Kaiser Permanente Northern California. Given prior screening results, we analyzed how each child in the dataset would have been classified by several potential screening algorithms that differed with respect to information used and screening thresholds. Outcomes considered several perspectives, including population health, specialty care, primary care, and individual families.

Analyses supported a screening algorithm that: 1) includes all sources of screening information, 2) recommends evaluation for children with an estimated chance of receiving an autism diagnosis by 4 years of ≥50%, and 3) recommends active monitoring for children with an estimated probability of autism of ≥3.3% (but <50%). If implemented with fidelity, this algorithm could result in the direct referral of 1.8% of children, achieving 34.2% sensitivity and 62.1% positive predictive value before requiring further clinical decision-making. Tradeoffs with other potential screening algorithms are considered.

Based on results, we recommend that health systems use EMR data to consider tradeoffs between potential algorithms when implementing or refining screening protocols. We also recommend replication over time to account for secular changes.