Decipher-MR: a vision-language foundation model for 3D MRI representations

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Machine Learning

Background:

• Magnetic Resonance Imaging (MRI) is vital for clinical diagnosis and research.
• Complexity and data heterogeneity in MRI challenge scalable and generalizable machine learning.
• Existing foundation models struggle with MRI due to data scarcity and limited anatomical scope.

Purpose of the Study:

• To introduce Decipher-MR, a 3D MRI-specific vision-language foundation model.
• To address limitations in applying foundation models to the unique characteristics of MRI data.
• To create a robust and reusable AI foundation for diverse MRI applications.

Main Methods:

• Trained Decipher-MR on 200,000 MRI series from over 22,000 studies.
• Integrated self-supervised vision learning with report-guided text supervision.
• Employed a modular design for efficient tuning of task-specific decoders with a frozen encoder.

Main Results:

• Decipher-MR demonstrated consistent improvements across disease classification, demographic prediction, anatomical localization, and cross-modal retrieval.
• Outperformed existing foundation models and task-specific approaches in evaluated tasks.
• Showcased the model's ability to build robust representations for broad MRI applications.

Conclusions:

• Decipher-MR serves as a promising and reusable foundation for AI in MRI.
• The model's performance suggests significant potential for advancing MRI-based diagnostics and research.
• The findings support the broader application of foundation models in medical imaging analysis.