DuA: Dual Attentive Transformer in Long-Term Continuous EEG Emotion Analysis

Affective brain-computer interfaces (aBCIs) are increasingly recognized for their potential in monitoring and interpreting emotional states through electroencephalography (EEG) signals. Current EEG-based emotion recognition methods perform well with short segments of EEG data (known as segment-based emotion analysis). However, these methods encounter significant challenges in real-life scenarios where emotional states evolve over extended periods. To address this issue, we propose a novel Dual Attentive (DuA) transformer framework for long-term continuous EEG emotion analysis. Unlike segment-based approaches, our model processes an entire EEG trial as a whole, identifying emotions at the trial level, referred to as trial-based emotion analysis. This framework is designed to adapt to varying signal lengths, providing a substantial advantage over traditional methods. The DuA transformer incorporates three key modules: the spatial-spectral network module, the temporal network module, and the transfer learning module. The spatial-spectral network module simultaneously captures spatial and spectral information from EEG signals, while the temporal network module detects temporal dependencies within long-term EEG data. The transfer learning module enhances the model's adaptability across different subjects and conditions. To verify the effectiveness of the proposed DuA transformer, we construct a long-term continuous EEG emotion database and extensively evaluate our model using the self-constructed database along with two benchmark EEG emotion databases. On the basis of the trial-based leave-one-subject-out cross-subject cross-validation protocol, our experimental results demonstrate that DuA significantly outperforms existing methods in long-term continuous EEG emotion analysis, with an average improvement of 2.8%. The DuA transformer's ability to adapt to varying signal lengths and its superior performance across diverse subjects and conditions highlight its potential for real-world applications, enhancing the overall user experience and efficacy of aBCI systems.