Related Experiment Video
Updated: Jun 4, 2025
11:28
Concurrent EEG and Functional MRI Recording and Integration Analysis for Dynamic Cortical Activity Imaging
Published on: June 30, 2018
11.6K
Multi-modal cross-domain self-supervised pre-training for fMRI and EEG fusion
Xinxu Wei1, Kanhao Zhao2, Yong Jiao2
1Department of Electrical and Computer Engineering, Lehigh University, Bethlehem, PA 18015, USA.
Summary
This study introduces a novel Multi-modal Cross-domain Self-supervised Pre-training Model (MCSP) to integrate functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) data. The MCSP model effectively combines information across domains and modalities for improved brain disorder detection.
Area of Science:
- Neuroscience
- Medical Imaging
- Artificial Intelligence
Background:
- Functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) detect brain abnormalities.
- Current methods often analyze single modalities, missing complementary information crucial for understanding disorder pathology.
Purpose of the Study:
- To develop a novel Multi-modal Cross-domain Self-supervised Pre-training Model (MCSP).
- To synergize multi-modal neuroimaging data across spatial, temporal, and spectral domains for enhanced brain disorder analysis.
Main Methods:
- Proposed the MCSP model utilizing self-supervised learning.
- Implemented cross-domain self-supervised loss with domain-specific augmentation and contrastive loss.
- Introduced cross-modal self-supervised loss for fMRI and EEG data fusion and feature convergence.
Main Results:
- Constructed a large-scale pre-training dataset and pretrained the MCSP model.
- Demonstrated superior performance and generalizability of the MCSP model on multiple classification tasks.
- Successfully integrated cross-domain features from fMRI and EEG.
Conclusions:
- The MCSP model represents a significant advancement in fusing fMRI and EEG data.
- This novel integration of cross-domain features enhances neuroimaging research, particularly for mental disorders.
- The self-supervised approach effectively leverages multimodal neuroimaging data for improved diagnostic capabilities.
