A spatial and temporal transformer-based EEG emotion recognition in VR environment.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Deep learning has advanced Electroencephalograph (EEG) emotion recognition for brain-computer interfaces.
• Current EEG emotion recognition models often lack ecological validity due to laboratory-based emotion induction and testing.
• Virtual Reality (VR) offers a realistic environment for more ecologically valid emotional research.

Purpose of the Study:

• To develop and validate a novel, purely Transformer-based method for EEG emotion recognition.
• To assess the model's effectiveness in both traditional laboratory settings and immersive VR environments.
• To demonstrate the model's ability to generalize across different emotion elicitation paradigms.

Main Methods:

• Collected EEG data from participants viewing VR videos.
• Proposed EmoSTT, a Transformer-based architecture utilizing two modules for temporal and spatial EEG signal analysis.
• Validated EmoSTT on both a passive laboratory paradigm and an active VR paradigm dataset.

Main Results:

• EmoSTT achieved robust emotion classification performance compared to state-of-the-art methods.
• The proposed method demonstrated effective transferability between different emotion elicitation paradigms.
• The Transformer-based approach successfully modeled both temporal and spatial information in EEG signals.

Conclusions:

• EmoSTT offers a promising solution for more ecologically valid EEG emotion recognition.
• The model's ability to generalize across paradigms enhances its real-world applicability.
• Transformer networks are effective for capturing complex spatio-temporal dynamics in EEG for emotion recognition.