MT-RCAF: A Multi-Task Residual Cross Attention Framework for EEG-based emotion recognition and mood disorder detection
Related Experiment Video
Updated: May 21, 2025
Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
Published on: May 15, 2016
Xinni Kong1, Yaru Guo1, Yu Ouyang1
1School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, Zhejiang, China.
Background And Objective:
Prolonged abnormal emotions can gradually evolve into mood disorders such as anxiety and depression, making it critical to study the relationship between emotions and mood disorders to explore the causes of mood disorders. Existing research on EEG-based emotion recognition and mood disorder detection typically treats these two tasks separately, missing potential synergies between them. The purpose is to reveal the relationship between emotions and mood disorders and propose a Multi-Task Residual Cross Attention Framework (MT-RCAF) to enhance both classification performances.
Methods:
In MT-RCAF, the Feature Extraction module extracts specific and shared features for the corresponding tasks. The Residual Multi-head Cross Attention (RMCA) module dynamically adjusts attention weights to explicitly capture both shared and task-specific information, enhancing complementarity and feature sharing. The Gated Multi-embedding (GME) module filters out irrelevant features, improving task-specific performance. Finally, the Task Tower Classification module balances losses across tasks to facilitate both emotion recognition and mood disorder detection.
Results:
We conducted experiments on the DEAP dataset as well as the self-collected Emotion and Mood Disorder Dataset (EMDD) to validate the effectiveness of MT-RCAF. The results show that the framework gains improvement in strongly correlated task groups, with average accuracy increases of 3.22% for emotion recognition and 3.91% for mood disorder detection, and in generally correlated task groups, with average accuracy increases of 2.87% for valence and 3.34% for arousal. The study also reveals that mood disorders (depression or anxiety) increase sensitivity to negative emotions, and intense emotions enhance mood disorder detection.
Conclusion:
The study validates the relationship between emotions and mood disorders from a deep-learning perspective and finds that interconnected tasks result in more accurate and robust results.
