A vision-language foundation model for Alzheimer's disease diagnosis using MRI and clinical data

Reliable early detection of Alzheimer's disease (AD) remains difficult due to heterogeneous progression trajectories and variability in clinical presentation. Multimodal approaches leveraging neuroimaging and clinical data offer promise but often struggle with effective integration and generalization.

We developed Alzheimer's Disease Language and Image Pre-Training (ADLIP), a vision-language framework that integrates 3D T1-weighted magnetic resonance imaging with structured clinical records. The model uses a multi-teacher training strategy to enhance generalizability and robustness across modalities, enabling more reliable representation learning for AD diagnosis.

ADLIP outperformed baseline CLIP and fine-tuned CLIP models in three-class classification and zero-shot diagnosis, achieving improved accuracy, F₁ score, and macro-area under the curve. Longitudinal evaluation demonstrated alignment with clinical trajectories. ADLIP maintained performance across racial subgroups without retraining.

Our results demonstrate that contrastive multimodal representation learning enables clinically meaningful, generalizable, and temporally stable AD diagnosis across diverse populations.

A novel vision-language foundation model (Alzheimer's Disease Language and Image Pre-Training [ADLIP]) integrates 3D magnetic resonance imaging and clinical text for Alzheimer's disease diagnosis. ADLIP enables zero-shot prediction of unseen data and cognitive scores without task-specific fine-tuning. The model demonstrates strong generalizability across racially diverse cohorts, supporting equitable clinical use. Longitudinal evaluation shows alignment with disease progression, highlighting utility for monitoring applications.