Developing an integrated algorithm to support autism diagnostic decisions: Model performance and statistical fairness

Area of Science:

• Neuroscience
• Computer Science
• Developmental Psychology

Background:

• Autism spectrum disorder (ASD) diagnosis relies on integrating diverse information, including observations, interviews, and informant ratings.
• Machine learning (ML) shows promise in synthesizing complex datasets for improved diagnostic classification.
• This study evaluated an ML algorithm's ability to weight multiple diagnostic information sources for ASD.

Purpose of the Study:

• To assess the efficacy of a machine learning algorithm in supporting autism spectrum disorder diagnosis.
• To determine if ML can appropriately weight various sources of diagnostically relevant information.
• To evaluate the statistical fairness of the developed ML algorithm across different demographic groups.

Main Methods:

• Telehealth diagnostic assessments were conducted for 639 toddlers receiving early intervention.
• Data included the Toddler Autism Symptom Interview (TASI), TELE-ASD-PEDS (TAP), and caregiver/provider rating scales.
• An elastic net regularized regression model was developed and validated, with statistical fairness assessed.

Main Results:

• The ML algorithm incorporated rating scales, TASI, and TAP items, achieving high performance (sensitivity >0.90, specificity >0.75).
• Prevalence of autism in the sample was 80.4%.
• While overall performance was high, statistical fairness varied, with more false positives in advantaged groups, suggesting potential under-classification in minoritized groups.

Conclusions:

• Machine learning algorithms can effectively integrate multiple data sources to aid in autism diagnosis.
• The developed algorithm demonstrated high diagnostic accuracy but highlighted the need for careful bias evaluation.
• Expert clinical judgment remains essential and should not be replaced by ML algorithms in ASD diagnosis.