EEG Emotion Recognition Supervised by Temporal Features of Video Stimuli

Area of Science:

• Neuroscience
• Computer Science
• Artificial Intelligence

Background:

• Electroencephalography (EEG) signals capture brain activity for emotion recognition.
• Accurate subject-independent emotion recognition from EEG is challenging due to signal non-stationarity and low signal-to-noise ratio.
• Integrating external data can potentially improve EEG-based emotion recognition.

Purpose of the Study:

• To develop an end-to-end framework for emotion recognition by supervising EEG features with external video features.
• To leverage the complementarity of video stimuli and EEG signals for enhanced emotional representations.
• To address the challenges of subject-independent emotion recognition.

Main Methods:

• An end-to-end framework combining EEG and video feature extractors was proposed.
• A cross-modal transformer was employed to align EEG and video feature distributions.
• A self-attention mechanism was utilized for feature fusion.
• Experiments were conducted on DEAP and self-collected datasets.

Main Results:

• Supervising EEG features with stimulus information significantly enhanced emotion recognition performance.
• The proposed framework demonstrated reliable subject-independent emotion recognition.
• The integration of video evoked features proved effective in improving EEG signal analysis for emotions.

Conclusions:

• Enhancing EEG features with external stimulus information is a viable strategy for subject-independent emotion recognition.
• The proposed cross-modal framework effectively fuses EEG and video data for robust emotion recognition.
• This approach offers a promising direction for advancing affective computing and brain-computer interfaces.