A transformer-based unified multimodal framework for Alzheimer's disease assessment

Area of Science:

• Artificial Intelligence
• Neuroscience
• Medical Imaging

Background:

• Traditional deep learning for Alzheimer's disease (AD) uses separate methods for different data types.
• A unified approach is needed to analyze diverse data modalities for improved AD assessment.

Purpose of the Study:

• To introduce AD-Transformer, a novel unified deep learning model for Alzheimer's disease assessment.
• To integrate structural magnetic resonance imaging (sMRI), clinical, and genetic data for enhanced diagnostic accuracy.

Main Methods:

• Developed AD-Transformer, a transformer-based model integrating sMRI, clinical, and genetic data.
• Utilized Patch-CNN for image tokenization and linear projection for non-image data tokenization.
• Employed a transformer block to learn cross-modal representations from the ADNI database (1651 subjects).

Main Results:

• AD-Transformer achieved an average AUC of 0.993 for AD diagnosis and 0.845 for Mild Cognitive Impairment (MCI) conversion prediction.
• Outperformed traditional image-only and non-unified multimodal models in diagnostic accuracy.
• Demonstrated effective integration and analysis of multimodal data for AD assessment.

Conclusions:

• AD-Transformer offers a powerful, unified framework for AD diagnosis and MCI conversion prediction.
• The model effectively leverages multimodal data, paving the way for more precise clinical assessments.
• Presents a clinically adaptable strategy for utilizing diverse data in Alzheimer's disease research.