Emotion recognition based on multimodal physiological electrical signals.

Area of Science:

• Neuroscience and Computational Psychology
• Biomedical Signal Processing
• Artificial Intelligence in Healthcare

Background:

• Increasing mental health issues necessitate advanced diagnostic and intervention tools.
• Accurate emotion recognition is crucial for personalized mental health management.
• Existing methods often rely on single physiological signals, limiting accuracy.

Purpose of the Study:

• To develop a multimodal emotion recognition system fusing electroencephalography (EEG) and electrocardiography (ECG) signals.
• To accurately classify emotional states across three dimensions: potency, arousal, and dominance.
• To enhance emotion recognition accuracy and robustness using a deep learning approach.

Main Methods:

• A composite neural network model (Att-1DCNN-GRU) integrating attention mechanisms and gated recurrent units was designed.
• Extraction of time-domain, frequency-domain, and nonlinear features from EEG and ECG signals.
• Feature filtering using a Random Forest approach to improve model performance.

Main Results:

• The multimodal fusion model achieved high classification accuracy for potency, arousal, and dominance dimensions on the DREAMER dataset, reaching 95.95% for the 'value' dimension.
• Significant performance improvement compared to unimodal EEG or ECG recognition methods.
• Demonstrated strong cross-dataset generalization ability on the DEAP dataset.

Conclusions:

• Multimodal signal fusion offers substantial advantages for robust and accurate emotion recognition.
• The Att-1DCNN-GRU model shows significant potential for emotion computing and mental health management.
• Deep learning techniques are effective for processing complex physiological signals in emotion recognition tasks.