Echo-Vision-FM: a pre-training and fine-tuning framework for echocardiogram video vision foundation model

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Cardiovascular Diagnostics

Background:

• Echocardiogram analysis is crucial for cardiovascular diagnostics but often relies on manual interpretation or complex annotation processes.
• Developing robust and transferable representations from unlabeled echocardiogram videos is a significant challenge.
• Existing methods may require extensive manual annotation or lack generalizability across different datasets and clinical scenarios.

Purpose of the Study:

• To introduce Echo-Vision-FM, a self-supervised video learning framework for pre-training echocardiogram video encoders.
• To generate robust and transferable video representations from large-scale, unlabeled echocardiogram data.
• To enhance downstream performance in various echocardiogram analysis tasks, including diagnosis and morphological value estimation.

Main Methods:

• Utilized a self-supervised masked auto-encoding strategy with an 85% mask ratio on the MIMIC-IV-ECHO dataset.
• Introduced Spatial-Temporal Fusion Network (STF-Net) to integrate spatial and temporal correlations in video representations.
• Pre-trained an echo-video encoder without manual annotations, leveraging large-scale unlabeled data.

Main Results:

• Achieved high accuracy (0.905), F1 score (0.941), and AUC (0.931) for heart function diagnosis on the EchoNet-Dynamic dataset.
• Demonstrated strong performance in aortic stenosis diagnosis (AUC of 0.849) on the TMED dataset.
• Outperformed state-of-the-art models in cardiac morphological value estimation, including left ventricular ejection fraction (LV_EF) prediction (MAE of 3.87%, r² of 0.825) and volume estimations (r² values of 0.782 and 0.742).
• Achieved a Pearson correlation coefficient of 86.49% for LV_EF estimation on the CAMUS dataset, surpassing traditional segmentation-based methods.

Conclusions:

• Echo-Vision-FM provides a powerful and scalable approach for echocardiogram analysis, significantly advancing clinical diagnostics and research.
• The framework demonstrates robust cross-institutional generalizability and data efficiency, particularly in low-resource settings.
• The integration of STF-Net consistently improved performance across various tasks, highlighting its effectiveness in capturing spatial-temporal dynamics.