A Multimodal Dataset for Mixed Emotion Recognition.

Area of Science:

• Affective computing
• Human-computer interaction
• Psychophysiology

Background:

• Mixed emotions are increasingly recognized but underrepresented in multimodal datasets.
• Existing research lacks comprehensive datasets for mixed emotion recognition using physiological and visual signals.

Purpose of the Study:

• To introduce a novel multimodal dataset for mixed emotion recognition.
• To facilitate advancements in affective computing for complex emotional states.
• To validate emotion induction and classification methods using multimodal data.

Main Methods:

• Developed a rule-based video filtering system for effective emotion induction.
• Collected multimodal data (EEG, GSR, PPG, facial videos) from 73 participants.
• Recorded subjective emotional ratings using PANAS, VAD, and amusement-disgust scales.

Main Results:

• Technical validation confirmed effective emotion induction and classification.
• Support Vector Machine (SVM) with features from all modalities achieved 80.96% accuracy for 3-class emotion classification (positive, negative, mixed).

Conclusions:

• The presented multimodal dataset supports research in mixed emotion recognition.
• Multimodal signals, including physiological and facial data, show significant potential for identifying mixed emotional states.
• This work advances the field of affective computing by providing a valuable resource and demonstrating classification feasibility.