EEG-fNIRS Cross-Subject Emotion Recognition Based on Attention Graph Isomorphism Network and Contrastive Learning.

Area of Science:

• Neuroscience
• Affective Computing
• Machine Learning

Background:

• Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) capture brain dynamics for emotion recognition.
• Combining EEG and fNIRS shows promise but faces challenges in multi-modal fusion and cross-subject generalization due to signal heterogeneity and inter-subject variability.

Purpose of the Study:

• To develop a robust method for EEG-fNIRS based emotion recognition that overcomes limitations in cross-modal fusion and cross-subject generalization.
• To introduce DC-AGIN, a dual-contrastive learning attention graph isomorphism network designed to enhance emotion recognition accuracy and generalizability.

Main Methods:

• Proposed DC-AGIN model utilizing an attention-weighted Graph Isomorphism Network (AGIN) encoder for adaptive feature emphasis.
• Implemented cross-modal contrastive learning to align EEG and fNIRS representations in a shared semantic space.
• Employed supervised contrastive learning to reduce subject-specific information and promote subject-invariant affective representations.

Main Results:

• Achieved 96.98% accuracy in subject-dependent four-class emotion classification.
• Demonstrated significant cross-subject generalization with 62.56% accuracy under a leave-one-subject-out (LOSO) protocol.
• Outperformed existing models, achieving state-of-the-art (SOTA) performance in EEG-fNIRS emotion recognition.

Conclusions:

• Attention aggregation, cross-modal, and cross-subject contrastive learning significantly improve the robustness of EEG-fNIRS emotion recognition.
• The DC-AGIN model effectively learns generalizable emotion representations, demonstrating its potential for real-world affective computing applications.