MBRSTCformer: a knowledge embedded local-global spatiotemporal transformer for emotion recognition

Emotion recognition is an essential prerequisite for realizing generalized BCI, which possesses an extensive range of applications in real life. EEG-based emotion recognition has become mainstream due to its real-time mapping of brain emotional activities, so a robust EEG-based emotion recognition model is of great interest. However, most existing deep learning emotion recognition methods treat the EEG signal as a whole feature extraction, which will destroy its local stimulation differences and fail to extract local features of the brain region well. Inspired by the cognitive mechanisms of the brain, we propose the multi-brain regions spatiotemporal collaboration transformer (MBRSTCfromer) framework for EEG-based emotion recognition. First, inspired by the prior knowledge, we propose the Multi-Brain Regions Collaboration Network. The EEG data are processed separately after being divided by brain regions, and stimulation scores are presented to quantify the stimulation produced by different brain regions and feedback on the stimulation degree to the MBRSTCfromer. Second, we propose a Cascade Pyramid Spatial Fusion Temporal Convolution Network for multi-brain regions EEG features fusion. Finally, we conduct comprehensive experiments on two mainstream emotion recognition datasets to validate the effectiveness of our proposed MBRSTCfromer framework. We achieved 98.63 $\%$ , 98.15 $\%$ , and 98.58 $\%$ accuracy on the three dimensions (arousal, valence, and dominance) on the DEAP dataset; and 97.66 $\%$ , 97.07 $\%$ , and 97.97 $\%$ on the DREAMER dataset.