Exploring multimodal large language models on transthoracic Echocardiogram (TTE) tasks for cardiovascular decision...

Area of Science:

• Artificial Intelligence in Medicine
• Cardiovascular Imaging Analysis
• Machine Learning for Healthcare

Background:

• Multimodal large language models (LLMs) present novel opportunities for advancing cardiovascular decision support systems.
• Interpreting complex echocardiographic data remains a critical area for AI-driven enhancement in clinical practice.

Purpose of the Study:

• To systematically evaluate and benchmark diverse foundation models on echocardiogram-based tasks.
• To assess the effectiveness, limitations, and clinical potential of LLMs in cardiovascular applications.

Main Methods:

• Three cardiovascular imaging datasets (EchoNet-Dynamic, TMED2, TTE) were used for evaluating ejection fraction (EF) prediction, view classification, aortic stenosis (AS) assessment, and disease classification.
• Six multimodal LLMs, including cardiovascular-specific, medical-domain, and general-domain models, were tested using zero-shot, few-shot, and fine-tuning strategies.

Main Results:

• Domain-specific models like EchoClip showed strong zero-shot EF prediction (MAE 10.34).
• Fine-tuning significantly improved general-domain models, reducing MiniCPM-V 2.6's EF MAE to 31.93 and boosting view classification accuracy to 63.05%.
• Cardiovascular-focused and fine-tuned models outperformed others, particularly in EF estimation, though performance varied across tasks.

Conclusions:

• Domain-specific pretraining and model adaptation are crucial for effective AI in cardiovascular decision support.
• Cardiovascular-focused and fine-tuned LLMs demonstrate superior performance for complex tasks like EF estimation.
• These findings provide essential insights for integrating AI into clinical cardiovascular medicine.