Towards a cardiovascular magnetic resonance foundation model for multi-task cardiac image analysis

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Cardiovascular Imaging

Background:

• Cardiovascular magnetic resonance (CMR) analysis is complex, involving diverse image processing tasks.
• Foundation models (FM) show promise for automated image analysis in natural images (NI).
• A CMR-specific vision FM was developed to address these challenges.

Purpose of the Study:

• To develop and evaluate a CMR-specific vision foundation model.
• To finetune the FM for nine common CMR imaging tasks, including classification, segmentation, landmark localization, and pathology detection.
• To compare the FM's performance against state-of-the-art methods and a baseline model.

Main Methods:

• A Vision Transformer (ViT-S/8) was self-supervised pretrained using DINO on 36 million CMR images.
• The pretrained model was finetuned for nine CMR tasks using diverse datasets.
• Performance was evaluated against state-of-the-art methods and a baseline, with analyses on pretraining strategy, generalization, and few-shot learning.

Main Results:

• The CMR-specific FM achieved comparable or improved performance across most tasks compared to existing methods.
• The model outperformed the baseline, showing significant improvements in cine view classification (6.8% pp) and disease detection (14% pp).
• The FM demonstrated better generalization and few-shot performance, highlighting the importance of the pretraining strategy.

Conclusions:

• A specialized vision FM for medical imaging enhances accuracy and robustness over natural image FMs.
• Self-supervised pretraining provides an efficient, unified framework for CMR assessment.
• This approach accelerates the development of deep learning solutions for CMR image analysis, even with limited annotated data.