K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations

Area of Science:

• Affective computing
• Human-computer interaction
• Social signal processing

Background:

• Recognizing emotions in social interactions is crucial for applications using mobile sensors.
• Existing emotion datasets lack naturalistic data, hindering the study of 'in-the-wild' emotions.
• Constrained data collection environments limit the ecological validity of current emotion recognition research.

Purpose of the Study:

• To introduce K-EmoCon, a novel multimodal dataset for naturalistic affective interaction research.
• To facilitate the study of idiosyncratic emotions during spontaneous social exchanges.
• To enable multiperspective emotion assessment in realistic conversational contexts.

Main Methods:

• Collected multimodal data (audiovisual, EEG, physiological signals) using off-the-shelf devices.
• Recorded 16 sessions of 10-minute paired debates on social issues.
• Annotated continuous emotions (arousal-valence, 18 categories) from self, partner, and observer perspectives every 5 seconds.

Main Results:

• Developed K-EmoCon, the first publicly available dataset for multiperspective emotion analysis in social interactions.
• Captured rich, continuous emotional dynamics during naturalistic debates.
• Provided comprehensive annotations reflecting diverse viewpoints on emotional displays.

Conclusions:

• K-EmoCon addresses the critical need for naturalistic affective interaction data.
• The dataset supports advanced research in emotion recognition, particularly in social contexts.
• Multiperspective annotations offer unprecedented insights into subjective and objective emotional experiences.