Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism

Area of Science:

• Computer Vision
• Behavioral Science
• Machine Learning

Background:

• The Facial Action Coding System (FACS) is crucial for linking facial behavior to mental health but is labor-intensive and costly.
• Automated Action Unit (AU) detection faces limitations in accuracy and coverage, hindering comprehensive facial expression analysis.
• Existing methods struggle to represent the entirety of facial expressions, excluding many AUs.

Purpose of the Study:

• To introduce Facial Basis, a novel, data-driven coding system for facial movement analysis.
• To overcome the limitations of automated FACS coding, including manual annotation, limited movement repertoire, and non-additive unit combinations.
• To provide a comprehensive and unsupervised approach for deconstructing facial expressions in videos.

Main Methods:

• Developed a data-driven coding system, Facial Basis, with units representing localized, interpretable 3D facial movements.
• Implemented an unsupervised learning framework, eliminating the need for manual annotation.
• Ensured Facial Basis reconstructs all observable facial movements and that its units are additive.

Main Results:

• Facial Basis accurately reconstructs all observable facial movements, unlike automated FACS which uses a limited set of AUs.
• The system is unsupervised, bypassing the need for costly and time-consuming manual annotation.
• Facial Basis units are additive, overcoming limitations of non-additive AU combinations in existing methods.
• Facial Basis outperformed the leading AU detector in predicting autism diagnosis from video data.

Conclusions:

• Facial Basis presents a significant advancement as the first FACS alternative for deconstructing facial expressions into localized movements from video.
• The comprehensive and unsupervised nature of Facial Basis makes it a valuable tool for behavioral research.
• This method holds promise for improving our understanding of facial behavior in relation to mental health conditions like autism.