ST-SHAP: A hierarchical and explainable attention network for emotional EEG representation learning and decoding

Area of Science:

• Neuroscience
• Computer Science
• Human-Computer Interaction

Background:

• Emotion recognition using electroencephalogram (EEG) is a key area in human-computer interaction.
• Challenges remain in learning complex spatial-temporal representations and achieving explainable predictions from emotional EEG data.

Purpose of the Study:

• To propose a novel hierarchical and explainable attention network, ST-SHAP, for automatic emotional EEG classification.
• To combine the Swin Transformer (ST) with SHapley Additive exPlanations (SHAP) for improved performance and interpretability.

Main Methods:

• Generated 3D spatial-temporal features from emotional EEG data using frequency band filtering, temporal segmentation, spatial mapping, and interpolation.
• Employed a hierarchical attention network with Swin Transformer modules (W-MSA, SW-MSA, patch merging) for multiscale spatial-temporal representation learning.
• Utilized the SHAP method to identify important brain regions and enhance the explainability of the Swin Transformer model.

Main Results:

• Achieved high classification accuracies: 97.18% on the SEED dataset and 96.06% (arousal) and 95.98% (valence) on the DREAMER dataset in subject-dependent experiments.
• Identified important brain regions consistent with neurophysiological knowledge through a data-driven approach.
• Outperformed several existing methods in both subject-dependent and subject-independent emotional EEG decoding.

Conclusions:

• The proposed ST-SHAP algorithm demonstrates significant effectiveness and superiority in emotional EEG classification.
• The integration of Swin Transformer and SHAP provides a powerful and explainable approach for emotion recognition.