The Performance of Emotion Classifiers for Children With Parent-Reported Autism: Quantitative Feasibility Study

Area of Science:

• Artificial Intelligence
• Developmental Psychology
• Computational Linguistics

Background:

• Autism spectrum disorder (ASD) affects approximately 1 in 40 children, with increasing prevalence straining treatment access.
• Mobile AI solutions offer potential for autism therapy but may lack sufficient training for pediatric use.
• Facial and emotion detection AI models are readily available but require validation for specific populations.

Purpose of the Study:

• To evaluate the performance of commercial, off-the-shelf emotion classifiers on children with ASD.
• To assess the feasibility of using these AI models for developing mobile therapies for social communication deficits.
• To investigate the potential of crowdsourced data for training and validating AI in pediatric research.

Main Methods:

• A mobile game, "Guess What?", was used to elicit and record children's facial expressions during social interaction.
• 21 children with ASD participated, generating 2602 video frames of emotional expressions.
• Four state-of-the-art facial emotion classifiers were tested on the collected data.

Main Results:

• All tested emotion classifiers demonstrated poor performance across most emotions, except for 'happy'.
• Accuracy rates for correctly identifying emotions were below 60.18% for all classifiers.
• Specific difficulties were noted in identifying 'angry' (11%) and 'disgust' (14%) emotions.

Conclusions:

• Commercial emotion classifiers are currently insufficiently trained for reliable use in autism treatment and tracking.
• Development of secure, privacy-preserving methods for increasing labeled training data is crucial.
• Improved AI model performance is necessary before widespread adoption in AI-enabled social therapies for ASD.