Emotion Recognition from Facial Expressions Considering Individual Differences in Emotional Intelligence

Area of Science:

• Psychology
• Computer Science
• Affective Computing

Background:

• Facial Expression Recognition (FER) in naturalistic settings faces challenges due to label ambiguity and inconsistent stimulus-response alignment.
• A data-centric approach is crucial for improving FER by considering qualitative factors in training data.

Purpose of the Study:

• To investigate the impact of emotional intelligence (EI)-stratified training data on FER performance.
• To treat EI as a factor influencing affective data consistency and its effect on FER.

Main Methods:

• Collected naturally elicited facial expressions in a controlled experiment with arousal and valence ratings.
• Grouped participants into High and Low EI based on subjective evaluations and affect estimator outputs.
• Trained binary classifiers for arousal and valence recognition using EI-stratified data and evaluated performance across different test sets.

Main Results:

• EI-stratified training showed context-dependent performance differences, particularly in baseline and ambiguous conditions.
• No significant performance differences were observed under unambiguous conditions.
• Item-level analyses revealed significant classification correctness differences in specific task-condition combinations.

Conclusions:

• FER performance is influenced by both model architecture and the statistical coherence of training data.
• EI-informed data selection can enhance FER in ambiguity-prone naturalistic scenarios.
• Findings support the importance of data quality and structure in developing robust emotion recognition systems.