Vision-language foundation model for echocardiogram interpretation

Area of Science:

• Cardiovascular Imaging
• Artificial Intelligence
• Medical Informatics

Background:

• Development of AI models for echocardiography is hindered by limited annotated clinical data.
• Existing models struggle with diverse patient populations and imaging indications.

Purpose of the Study:

• To develop EchoCLIP, a vision-language foundation model for echocardiography.
• To improve the performance of cardiac imaging AI by learning image-text relationships.

Main Methods:

• Trained EchoCLIP on over 1 million cardiac ultrasound videos and expert interpretations.
• Utilized a long-context variant (EchoCLIP-R) with a custom tokenizer for advanced analysis.

Main Results:

• EchoCLIP accurately predicts left ventricular ejection fraction (MAE 7.1%) and identifies intracardiac devices (AUC up to 0.97).
• EchoCLIP-R identifies patients across videos (AUC 0.86), clinical transitions (AUC up to 0.79), and enables effective image-to-text search.

Conclusions:

• EchoCLIP demonstrates strong performance on diverse echocardiography benchmarks without task-specific training.
• This foundation model represents a significant advancement for preliminary interpretation in cardiovascular imaging.