Using Real-World Evidence to Estimate the Predictive Value of Screening Questionnaires: Autism Screening as a Case...

We used real-world evidence to estimate the accuracy of autism screening questionnaires, including sensitivity, specificity, and predictive probabilities for positive results and for individual screening scores.

This study utilized electronic medical records (EMR) from 33,490 families of children aged 4 years and under and enrolled in Kaiser Permanente Northern California. Independent variables included results of the Parents' Observations of Social Interactions (POSI) and Modified Checklist for Autism in Toddlers-Revised (M-CHAT-R), which were implemented as primary and secondary screeners, respectively. The primary outcome was an autism diagnosis in the EMR by age 4 years.

The POSI displayed 69.5% sensitivity and 76.4% specificity at 18 months, and 70.3% sensitivity and 80.8% specificity at 24 months. Predictive probabilities for positive scores varied; for example, 4.9% of children scoring 3 on the POSI at 24 months were diagnosed with autism by 4 years, compared to 76.8% of children scoring 7. Predictive values were higher among children who completed the MCHAT-R as a secondary screener, and even higher among those who had a positive MCHAT-R screen.

Demonstrating the range of information on screening that can be learned from EMR data, findings support the accuracy of the POSI while providing evidence of predictive probabilities to inform clinical care. Evidence and theory demonstrate that screening accuracy can be influenced by context. Thus, we recommend replication in other health systems to ensure fit-to-context and replication over time to identify potential trends in prevalence and decisions that may influence the utility of autism screening.