EmoTrans attention based emotion recognition using EEG signals and facial analysis with expert validation

Area of Science:

• Affective computing
• Human-computer interaction
• Neuroscience

Background:

• Traditional emotion recognition lacks ecological validity due to controlled stimuli.
• Understanding human emotions is crucial for advancing human-computer interaction.
• Physiological signals and facial expressions are key indicators of emotional states.

Purpose of the Study:

• To propose and validate the EmoTrans model for accurate emotion recognition.
• To integrate electroencephalogram (EEG) and facial video data for comprehensive emotional state analysis.
• To enhance the ecological validity of emotion recognition models.

Main Methods:

• Utilized the DEAP dataset comprising EEG recordings and facial videos from participants viewing movie clips.
• Integrated features from EEG signals (time, frequency, wavelet domains) and facial video data.
• Employed an attention-based architecture for feature prioritization and validated with machine learning, deep learning, and Leave-one-subject-out cross-validation (LOSO-CV).

Main Results:

• EmoTrans achieved high accuracies: 89.3% (arousal), 87.8% (valence), 88.9% (dominance), and 89.1% (liking).
• Demonstrated an overall 89% classification accuracy for various emotions like happiness, excitement, calmness, and distress.
• Statistical significance confirmed EmoTrans outperforms baseline models via paired t-test.

Conclusions:

• The EmoTrans model offers a robust and ecologically valid approach to emotion recognition.
• Integrating multi-modal data (EEG and facial analysis) significantly enhances emotion classification accuracy.
• The attention-based architecture effectively leverages relevant features for a nuanced understanding of human emotional states.