SFT-HN: a novel spatial-frequency-temporal hybrid network for EEG-based emotion recognition

Area of Science:

• Neuroscience
• Artificial Intelligence
• Signal Processing

Background:

• Electroencephalograph (EEG) emotion recognition is crucial for brain-computer interfaces (BCIs).
• Deep learning methods outperform traditional techniques in EEG emotion recognition.
• Challenges remain in fusing spatial, frequency, and temporal EEG information and utilizing discriminative local patterns.

Purpose of the Study:

• To propose a novel hybrid model, the Spatial-Frequency-Temporal Hybrid Network (SFT-HN), for enhanced EEG emotion recognition.
• To effectively fuse spatial, frequency, and temporal information from EEG signals.
• To leverage discriminative local patterns for improved emotion classification.

Main Methods:

• Developed a Spatial-Frequency-Temporal Hybrid Network (SFT-HN) incorporating Spatial Frequency Residual Modules (SFRM) and an attention-based Bidirectional Long Short-Term Memory (ATBI-LSTM).
• Utilized 4D representations of raw EEG signals to preserve spatial, frequency, and temporal information.
• Employed split-convert-merge techniques, residual, and attention mechanisms within SFRM for spatial-frequency feature extraction.
• Incorporated an attention mechanism in ATBI-LSTM to capture temporal dependencies.

Main Results:

• Achieved average accuracies of 97.61% (arousal) and 97.57% (valence) on the DEAP dataset.
• Attained an average accuracy of 97.44% on the SEED dataset.
• Demonstrated robust generalization with an average accuracy of 96.24% on the novel FACED dataset.

Conclusions:

• The SFT-HN model effectively integrates spatial, frequency, and temporal EEG features for superior emotion recognition.
• The proposed model demonstrates high accuracy and robust generalization across multiple datasets.
• The SFT-HN offers a promising advancement in EEG-based emotion recognition for BCIs.