Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data

Area of Science:

• Artificial Intelligence in Medicine
• Machine Learning for Healthcare
• Neuroscience and Medical Imaging

Background:

• Accurate medical classification relies on multi-modal data, but existing models struggle with missing data, leading to significant under-utilization.
• Deep learning performance is hampered by the scarcity of labeled medical images, especially when modalities are incomplete.
• A flexible multi-modal approach is needed to handle missing data in diverse clinical settings.

Purpose of the Study:

• To present the Multi-Modal Mixing Transformer (3MT), a novel deep learning model designed for disease classification using multi-modal data.
• To develop a method that effectively handles missing modalities, ensuring full data utilization.
• To evaluate 3MT's performance in classifying Alzheimer's Disease (AD) and predicting mild cognitive impairment (MCI) conversion.

Main Methods:

• Introduced the Multi-Modal Mixing Transformer (3MT), a transformer-based model for disease classification.
• Employed a novel Cascaded Modality Transformers architecture with cross-attention to integrate multi-modal information.
• Implemented a unique modality dropout mechanism to enhance robustness and independence against missing data.

Main Results:

• 3MT demonstrated state-of-the-art performance on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset for AD classification and MCI conversion prediction.
• The model achieved high performance on the Australian Imaging Biomarker & Lifestyle Flagship Study of Ageing (AIBL) dataset, even with missing data.
• The modality dropout mechanism ensured full data utilization in scenarios with missing modalities.

Conclusions:

• 3MT offers a versatile solution for multi-modal disease classification, adeptly handling missing data scenarios.
• The model's ability to mix arbitrary numbers of modalities and feature types enhances its applicability.
• 3MT represents a significant advancement in leveraging incomplete multi-modal data for improved medical classification and prediction.