MSTGC: Multi-Channel Spatio-Temporal Graph Convolution Network for Multi-Modal Brain Networks Fusion

Area of Science:

• Neuroscience
• Computational Biology
• Medical Imaging

Background:

• Multi-modal brain networks analyze brain region connectivity using structural and functional data.
• Existing fusion methods struggle to capture spatio-temporal and topological features effectively.
• Accurate analysis of brain networks is crucial for understanding and diagnosing neurological disorders.

Purpose of the Study:

• To develop an advanced framework for fusing multi-modal brain networks.
• To effectively extract spatio-temporal and topological characteristics from fused networks.
• To enhance the accuracy of brain disease diagnosis using multi-modal network analysis.

Main Methods:

• An adaptive multi-channel graph convolution network (GCN) fusion framework with graph contrast learning was developed.
• Dynamic brain network representations were created using overlapping time windows of ROI-based signals.
• Contrastive constraints (multi-modal fusion InfoMax, inter-channel InfoMin) and stacked long short-term memory (LSTM) units were employed to extract features.

Main Results:

• The proposed framework effectively mines complementary and discriminative features from multi-modal brain networks.
• Dynamic spatio-temporal characteristics and topological structures were successfully captured.
• Experiments on an epilepsy dataset demonstrated superior performance compared to existing state-of-the-art methods.

Conclusions:

• The developed adaptive multi-channel GCN fusion framework offers a powerful approach for analyzing multi-modal brain networks.
• The method shows significant potential for improving the accuracy and efficacy of brain disease diagnosis.
• This work advances the field of neuroimaging analysis by integrating spatio-temporal dynamics and topological information.