Mixture-of-Experts Electroencephalography-Functional Magnetic Resonance Imaging Fusion for Interpretable...

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide complementary information about brain function, and have shown significant promise in detecting functional abnormalities in various brain disorders. However, the distinct signal characteristics and disparate representational spaces of EEG and fMRI pose severe challenges to effective multimodal fusion, thereby hindering accurate computer-assisted diagnosis of brain disorders with conventional fixed models. This study presents Brain Mixture-of-Experts (BrainMoE), an adaptive and interpretable EEG-fMRI fusion framework for computer-assisted diagnosis of brain disorders, integrating multimodal brain features through modality-specific experts, a shared neural-state expert, and an adaptive routing mechanism. BrainMoE first projects EEG and fMRI signals into a unified Desikan-Killiany (DK) atlas region-of-interest (ROI) space, then uses graph encoders to extract modality-specific brain-network representations. The soft-routing module produces a routing representation, and the Expert Gate in the Fuse Module generates sample-specific weights to combine the EEG, fMRI, and shared neural-state expert representations. To handle incomplete acquisition scenarios, modality-state masks and missing-modality tokens are incorporated, allowing the same trained model to perform full EEG-fMRI, EEG-only, and fMRI-only inference. Finally, node-occlusion analysis provides ROI-level attribution maps for both EEG- and fMRI-derived predictions. The framework was evaluated on the Healthy Brain Network (HBN) dataset across five binary brain disorder classification tasks, including major depressive disorder, anxiety disorder, reading impairment, autism spectrum disorder, and attention-deficit/hyperactivity disorder. BrainMoE outperformed state-of-the-art comparison algorithms, achieving an average AUC of 86.9 ± 3.0%, and ablation experiments supported the contribution of the routing and expert-fusion components. Furthermore, the interpretability analysis identifies group-level ROI contributions to disease classification that are consistent with previously reported neuroimaging findings. This method supports computer-assisted diagnosis of brain disorders by addressing the challenge of integrating heterogeneous EEG-fMRI neural representations while maintaining interpretability and inference under simulated missing-modality conditions.