DSP-MCF: dual stream pre-training and multi-view consistency fine-tuning for cross-subject EEG emotion recognition

Electroencephalogram (EEG) emotion recognition is attracting increasing attention in the field of brain-computer interface due to its strong objectivity and non-forgery. However, cross-subject emotion recognition is complicated by individual variability, limited availability of EEG data, and interference in certain channels during EEG acquisition.

We propose a novel synergistic Dual Stream Pre-training and Multi-view Consistency Fine-tuning (DSP-MCF) framework. The DSP-MCF is based on a domain generalization architecture. The framework includes a dual stream pre-training stage, wherein the spatiotemporal encoder-decoder network extracts generalized spatiotemporal representations from masked channels and reconstructs EEG features from incomplete data. Then, a multi-view consistency loss function is proposed during the multi-view consistency fine-tuning. This loss function is essential for aligning the distribution of emotion predictions derived from various perspectives, specifically from actual and masked EEG data.

Experimental results demonstrate that the proposed DSP-MCF framework outperforms state-of-the-art methods in cross-subject EEG emotion recognition tasks. The model achieved an accuracy of 89.76% on the SEED dataset and 77.02% on the SEED-IV dataset.

The findings indicate that the DSP-MCF framework effectively addresses individual variability and maintains robust performance even under channel loss. By integrating spatiotemporal reconstruction with multi-view consistency, the model provides a reliable solution for handling incomplete or degraded EEG signals in practical BCI applications.