Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity

Area of Science:

• Psychology
• Computer Science
• Affective Computing

Background:

• Facial expressions are key to human communication, conveying emotions across diverse populations.
• Traditional research focused on discrete emotions, with less emphasis on affect intensity due to coding challenges.
• Manual coding of facial actions for affect intensity is labor-intensive and prone to reliability issues.

Purpose of the Study:

• To develop and validate computer-vision and machine learning (CVML) methods for analyzing facial expressions.
• To automate the coding of positive and negative affect intensity in large datasets.
• To understand the facial action patterns human judges use to rate affect intensity.

Main Methods:

• Utilized computer-vision and machine learning (CVML) on 4,648 video recordings from 125 participants.
• Correlated CVML-derived facial action patterns with affect intensity ratings from expert human coders.
• Employed interpretable machine learning to identify key facial actions for affect rating.

Main Results:

• CVML demonstrated strong correspondences with human affect intensity ratings.
• Identified the importance of specific facial actions in human affect intensity judgments.
• Successfully automated affect intensity coding for large-scale facial expression analysis.

Conclusions:

• CVML offers an efficient and reliable method for quantifying affect intensity from facial expressions.
• This approach can significantly advance research in affective computing and emotion recognition.
• CVML tools can provide insights into individual human perception of emotional expressions.