Unified Cross-Modal Attention-Mixer Based Structural-Functional Connectomics Fusion for Neuropsychiatric Disorder...

Area of Science:

• Neuroscience
• Artificial Intelligence
• Medical Imaging

Background:

• Understanding brain mechanisms is crucial for neuropsychiatric disorders like Schizophrenia.
• Traditional multimodal deep learning methods struggle to fully utilize complementary structural and functional connectomics data.
• Improved diagnostic performance requires advanced feature fusion techniques.

Purpose of the Study:

• To propose ConneX, a novel multimodal fusion method for integrating structural and functional brain connectomics data.
• To enhance the diagnostic performance for neuropsychiatric disorders by leveraging cross-modal interactions.
• To refine feature fusion using cross-attention mechanisms and MLP-Mixer.

Main Methods:

• Utilized modality-specific backbone graph neural networks (GNNs) for feature representation.
• Implemented a unified cross-modal attention network to fuse embeddings and capture interactions.
• Employed MLP-Mixer layers for refining global and local features and leveraging higher-order dependencies.
• Used a multi-head joint loss for end-to-end classification.

Main Results:

• ConneX demonstrated improved performance on two distinct clinical datasets.
• The framework effectively integrated structural and functional brain connectivity.
• Evaluations highlighted the robustness and enhanced diagnostic capabilities of the proposed method.

Conclusions:

• ConneX offers a significant advancement in multimodal fusion for brain connectomics.
• The method provides a robust approach for improved understanding and diagnosis of neuropsychiatric disorders.
• This study paves the way for more accurate diagnostic tools in neuroscience.