Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Imaging Studies for Cardiovascular System II:Types of Echocardiography

Imaging Studies for Cardiovascular System II:Types of Echocardiography

Echocardiography plays a role in assessing cardiac health and detecting heart conditions, with various types providing critical insights for diagnosis and treatment.
Types of Echocardiography
Transthoracic Echocardiography (TTE)
TTE is the most common type of echocardiogram which involves placing a transducer on the patient's chest, emitting sound waves to create heart images. TTE is invaluable for evaluating the heart's size, structure, and motion, making it particularly useful for...

Search research articles

Home
Research Domains
Information And Computing Sciences
Artificial Intelligence
Natural Language Processing
A Comparative Analysis Of Privacy-preserving Large Language Models For Automated Echocardiography Report Analysis.

Home
Research Domains
Information And Computing Sciences
Artificial Intelligence
Natural Language Processing
A Comparative Analysis Of Privacy-preserving Large Language Models For Automated Echocardiography Report Analysis.

Related Experiment Video

High-frequency High-resolution Echocardiography: First Evidence on Non-invasive Repeated Measure of Myocardial Strain, Contractility, and Mitral Regurgitation in the Ischemia-reperfused Murine Heart

High-frequency High-resolution Echocardiography: First Evidence on Non-invasive Repeated Measure of Myocardial Strain, Contractility, and Mitral Regurgitation in the Ischemia-reperfused Murine Heart

Published on: July 9, 2010

A comparative analysis of privacy-preserving large language models for automated echocardiography report analysis.

Elham Mahmoudi^1,2, Sanaz Vahdati¹, Chieh-Ju Chao^1,2

¹Department of Radiology, Radiology Informatics Lab, Mayo Clinic, Rochester, MN 55905, United States.

Journal of the American Medical Informatics Association : JAMIA

|May 7, 2025

View abstract on PubMed

Summary

This summary is machine-generated.

Open-source large language models (LLMs) show near-perfect accuracy in interpreting echocardiography reports for valvular heart disease (VHD) surveillance. Prompt optimization with chain of thought (CoT) significantly enhances performance, especially in larger models.

Keywords:

artificial intelligence information storage and retrieval natural language processing transthoracic echocardiography

More Related Videos

Ultrasonic Assessment of Myocardial Microstructure

Ultrasonic Assessment of Myocardial Microstructure

Published on: January 14, 2014

Evaluation of Left Ventricular Structure and Function using 3D Echocardiography

Evaluation of Left Ventricular Structure and Function using 3D Echocardiography

Published on: October 28, 2020

Related Experiment Videos

High-frequency High-resolution Echocardiography: First Evidence on Non-invasive Repeated Measure of Myocardial Strain, Contractility, and Mitral Regurgitation in the Ischemia-reperfused Murine Heart

High-frequency High-resolution Echocardiography: First Evidence on Non-invasive Repeated Measure of Myocardial Strain, Contractility, and Mitral Regurgitation in the Ischemia-reperfused Murine Heart

Published on: July 9, 2010

Ultrasonic Assessment of Myocardial Microstructure

Ultrasonic Assessment of Myocardial Microstructure

Published on: January 14, 2014

Evaluation of Left Ventricular Structure and Function using 3D Echocardiography

Evaluation of Left Ventricular Structure and Function using 3D Echocardiography

Published on: October 28, 2020

Area of Science:

Artificial Intelligence in Medicine
Cardiology Informatics
Natural Language Processing

Background:

Automated data extraction from echocardiography reports is crucial for large-scale registry creation and clinical surveillance of valvular heart diseases (VHD).
Evaluating the performance of open-source large language models (LLMs) guided by prompt instructions and chain of thought (CoT) is essential for this task.

Purpose of the Study:

To assess the efficacy of open-source LLMs in automating the interpretation of echocardiography reports for VHD surveillance.
To compare the performance of different LLMs with and without CoT prompting for classifying VHD severity and prosthetic valve presence.

Main Methods:

Utilized 200 random echocardiography reports from 2019 for prompt optimization and 1000 from 2023 for evaluation.

valvular heart disease

Five instruction-tuned LLMs were employed, guided by prompt instructions with and without CoT, to classify prosthetic valve presence and VHD severity.

Performance was evaluated using classification metrics and mean squared error (MSE) against expert-labeled ground truth.

Main Results:

LLMs employing CoT prompting achieved high accuracy: Llama3.0-70B (99.1% for VHD severity, 100% for prosthetic valve) and Qwen2.0 (98.9% for VHD severity, 99.9% for prosthetic valve).
Smaller models demonstrated lower accuracy for VHD severity (54.1%-85.9%) but maintained high accuracy for prosthetic valve detection (>96%).
CoT reasoning improved accuracy for larger models but increased processing time; errors were mainly due to irrelevant information or failure to follow instructions.

Conclusions:

Open-source LLMs demonstrate near-perfect performance for automated echocardiography report interpretation, facilitating VHD registry formation and surveillance.
Prompt optimization, particularly with CoT, is key for achieving high accuracy with larger models.
Balancing model performance with computational efficiency is critical for practical implementation in clinical settings.