Triad: Vision Foundation Model for 3D Magnetic Resonance Imaging

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Vision foundation models (VFMs) excel in diverse applications but often rely on 3D CT data, limiting their efficacy for 3D MRI due to inherent data differences.
• Existing VFMs pre-trained on CT scans may not generalize well to 3D Magnetic Resonance Imaging (MRI) due to distinct imaging principles and data characteristics.

Purpose of the Study:

• To develop and evaluate Triad, a VFM specifically designed for 3D MRI, addressing the limitations of CT-based models.
• To establish the largest 3D MRI pre-training dataset, Triad-131K, comprising 131,170 volumes.

Main Methods:

• Triad utilizes an autoencoder architecture for robust representation learning from the Triad-131K dataset.
• Organ-independent imaging descriptions were employed to guide the semantic distribution of the visual modality.
• Model performance was assessed across segmentation, classification, and registration tasks using 25 downstream datasets.

Main Results:

• Triad-initialized models demonstrated significant performance improvements across various tasks compared to scratch-trained counterparts.
• nnUNet-Triad improved segmentation by 2.51% (17 datasets), Swin-B-Triad enhanced classification by 4.04% (5 datasets), and SwinUNETR-Triad boosted registration by 4.00% (2 datasets).
• Pre-training benefits were most pronounced when data modalities and organs aligned between upstream and downstream tasks.

Conclusions:

• Triad represents a significant advancement in foundation models for 3D MRI, enhancing performance across critical clinical tasks.
• Large-scale pre-training on 3D MRI data is crucial for improving downstream task performance and reliability.
• Open-sourcing Triad's components aims to accelerate the adoption and development of robust 3D MRI foundation models in clinical practice.