Towards Context-Aware Facial Emotion Reaction Database for Dyadic Interaction Settings

Area of Science:

• Human-Computer Interaction
• Affective Computing
• Robotics

Background:

• Realistic facial behavior in social robots and AI agents is crucial for emotional credibility in human interactions.
• A key challenge is the scarcity of training data for human-to-human emotional interactions.
• Existing methods for collecting facial expression data in dyadic settings are insufficient.

Purpose of the Study:

• To develop a novel methodology for collecting a dyadic emotion reaction database.
• To capture genuine facial behaviors during face-to-face human interactions.
• To address the need for better training data in emotion recognition research.

Main Methods:

• Collected facial behavior data from 60 participants during dyadic interactions in emotionally primed contexts.
• Utilized webcam-based automatic facial expression capture.
• Analyzed collected data using three distinct Facial Expression Analysis (FEA) tools: iMotions, Mini-Xception, and Py-Feat.

Main Results:

• A new database of dyadic emotion reactions was successfully created.
• Comparative analysis revealed significant disagreement among the three FEA tools in predicting emotion reactions.
• Despite genuine reported emotions, automated analysis lacked consensus.

Conclusions:

• The study highlights the limitations of current FEA tools in dyadic settings.
• A more robust and effective model for emotion reaction prediction is necessary.
• This work contributes to developing adaptive behaviors for synthetic agents in human-computer interaction.