Interpretable Self-Supervised Facial Micro-Expression Learning to Predict Cognitive State and Neurological Disorders.

Area of Science:

• Neuroscience and Artificial Intelligence
• Human Behavior and Motor Systems

Background:

• Human behavior arises from complex motor system activity.
• Artificial intelligence (AI) can analyze facial muscle movements for insights into emotion, pain, and deception.
• Current AI methods often lack explainability and require extensive labeled data.

Purpose of the Study:

• To develop an explainable, self-supervised AI paradigm for learning temporal facial muscle movement patterns from limited data.
• To predict future speech behavior, specifically disfluency, in adults who stutter (AWS).

Main Methods:

• Proposed an explainable self-supervised representation-learning framework.
• Utilized facial muscle activity data from adults who stutter (AWS).
• Conducted empirical studies to validate predictive capabilities and explainability.

Main Results:

• Facial muscle movements around the eyes and lips significantly differ before fluent versus disfluent speech.
• The self-supervised approach demonstrated a minimum accuracy improvement of 2.51% over fully-supervised methods on the AWS dataset.
• Identified specific facial muscle patterns associated with speech disfluency.

Conclusions:

• Explainable self-supervised learning can effectively model temporal facial muscle movements for behavioral prediction.
• Facial muscle activity provides valuable biomarkers for speech disfluency in AWS.
• This approach offers a promising avenue for understanding and potentially mitigating speech disorders.