EEGMoE: A Domain-Decoupled Mixture-of-Experts Model for Self-Supervised EEG Representation Learning

Area of Science:

• Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models for electroencephalogram (EEG) analysis are often task-specific, limiting their generalizability.
• Current pretraining methods unify data formats but neglect crucial domain-specific details in EEG representations.
• Large-scale EEG data comprises diverse domains, necessitating methods that capture both shared and unique characteristics.

Purpose of the Study:

• To develop a self-supervised pretraining model for EEG representation learning that decouples domain-specific information.
• To learn both domain-shared and domain-specific representations from diverse EEG datasets.
• To improve the generalizability of EEG models across various tasks and datasets.

Main Methods:

• Propose EEG Mixture of Experts (EEGMoE), a Transformer-based domain-decoupled encoder utilizing a mixture-of-experts (MoE) block.
• Implement Top-K routing for specific expert groups and soft routing for shared expert groups within the MoE block.
• Pretrain EEGMoE on diverse EEG datasets and fine-tune/validate on emotion recognition, motor imagery classification, and mental workload detection tasks.

Main Results:

• EEGMoE outperforms state-of-the-art models on three public datasets for emotion recognition, motor imagery, and mental workload detection.
• Demonstrates strong generalization capabilities to new, unseen EEG domains.
• Extensive experiments and visualizations confirm the effectiveness of disentangled domain-specific representations.

Conclusions:

• EEGMoE successfully learns both shared and domain-specific EEG representations, enhancing model generalizability.
• The proposed domain-decoupled approach is crucial for leveraging the full potential of large-scale, multi-domain EEG data.
• The findings highlight the importance of disentangling representations for robust EEG analysis across various applications.