Related Experiment Video
Updated: May 16, 2025
09:47
Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
Published on: December 15, 2023
918
MM-GTUNets: Unified Multi-Modal Graph Deep Learning for Brain Disorders Prediction
IEEE Transactions on Medical Imaging
|April 1, 2025
Summary
This study introduces MM-GTUNets, a novel deep learning framework for predicting brain disorders (BDs) using multi-modal data. It enhances graph construction and cross-modal learning for improved diagnostic accuracy in large populations.
Area of Science:
- Neuroscience
- Artificial Intelligence
- Data Science
Background:
- Graph deep learning (GDL) shows promise for predicting population-based brain disorders (BDs) by integrating imaging and non-imaging data.
- Current GDL methods struggle with large-scale graphs and limited inter-modal data interactions, impacting predictive performance.
- Existing approaches often overlook complex correlations between different data modalities, leading to suboptimal brain disorder prediction outcomes.
Purpose of the Study:
- To propose MM-GTUNets, an end-to-end Graph Transformer-based multi-modal graph deep learning (MMGDL) framework for large-scale brain disorder prediction.
- To enhance the utilization of multi-modal disease-related information through dynamic graph construction and improved cross-modal feature learning.
- To overcome the limitations of existing GDL methods in handling large-scale data and complex inter-modal correlations for brain disorder diagnosis.
Main Methods:
- Developed Modality Reward Representation Learning (MRRL) with an Affinity Metric Reward System (AMRS) for dynamic population graph construction.
- Employed a variational autoencoder to align latent representations of non-imaging features with imaging features.
- Introduced Adaptive Cross-Modal Graph Learning (ACMGL) using a unified GTUNet encoder (combining Graph UNet and Graph Transformer) and a feature fusion module to capture modality-specific and shared features.
Main Results:
- MM-GTUNets demonstrated superior performance in diagnosing brain disorders on two public multi-modal datasets: ABIDE and ADHD-200.
- The framework effectively models multi-modal population graphs, addressing scalability issues inherent in GDL.
- Achieved improved capture of complex inter-modal correlations, leading to enhanced diagnostic accuracy for brain disorders.
Conclusions:
- MM-GTUNets offers a powerful and scalable solution for brain disorder prediction using multi-modal data.
- The proposed MRRL and ACMGL components significantly improve the integration and utilization of diverse data sources.
- This framework advances the application of graph deep learning in clinical neuroscience and brain disorder diagnostics.
