Multi-Domain Encoding of Spatiotemporal Dynamics in EEG for Emotion Recognition

Area of Science:

• Neuroscience
• Artificial Intelligence
• Affective Computing

Background:

• Mapping electroencephalogram (EEG) signals to emotional states (arousal-valence scores) is challenging due to complex spatial and temporal patterns.
• Existing methods struggle to capture the dynamic and context-dependent nature of brain activity during emotional experiences.

Purpose of the Study:

• To develop a hybrid EEG modeling method (MSDTTs) for accurately decoding human emotional states.
• To consider electrode connectivity and context dependency for improved emotional state modeling.
• To enhance the understanding of dynamic emotional behavior through advanced signal processing.

Main Methods:

• Designed a hybrid model named MSDTTs, integrating a Multi-domain Spatial Transformer (MST) and a Dynamic Temporal Transformer (DTT) with an attention mechanism.
• MST module extracts and fuses multi-domain spatial features from different brain regions.
• DTT module, incorporating Temporal Dynamic Excitation (TDE) and a multi-head convolutional transformer, extracts emotion-related dynamic temporal features and static context features.

Main Results:

• Achieved high classification accuracy: 98.91% (β-band, DEAP dataset), 97.52% (γ-band, SEED dataset), and 96.70% (γ-band, SEED-IV dataset).
• The proposed MSDTTs method demonstrated superior performance compared to state-of-the-art algorithms.
• Empirical experiments validated the effectiveness of the hybrid approach in emotion recognition.

Conclusions:

• The MSDTTs model effectively captures dynamic temporal and static spatial features from EEG signals for emotion recognition.
• The hybrid approach offers a significant advancement in accurately mapping EEG to emotional states.
• This method holds promise for applications in affective computing and brain-computer interfaces.