Voice-controlled super-resolution ultrasound imaging and reporting powered by multimodal large language models

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Ultrasound Technology

Background:

• Super-resolution ultrasound imaging (SRUI) offers advanced microvascular visualization but faces clinical adoption barriers due to complex workflows and interpretation challenges.
• Current SRUI methods require extensive parameter optimization and are time-consuming, limiting their practical application in clinical settings.

Purpose of the Study:

• To develop and evaluate a multimodal artificial intelligence (AI) framework to streamline super-resolution ultrasound imaging (SRUI) workflows.
• To integrate AI for natural language processing and image recognition to automate SRUI parameter selection, image reconstruction, and report generation.

Main Methods:

• A custom SRUI platform was integrated with DeepSeek-R1 (large language model for NLP) and MiniCPM-V (image recognition model).
• Clinicians used voice commands to initiate imaging, which were translated into acquisition parameters, including adaptive microbubble filtration determined by the Microbubble Similarity Score.
• The framework performed super-resolution reconstruction, extracted quantitative vascular metrics, and generated structured diagnostic reports.

Main Results:

• The AI framework successfully translated voice commands into SRUI acquisition parameters and generated structured diagnostic reports within approximately four minutes.
• Dynamic determination of filtration thresholds using the Microbubble Similarity Score optimized the imaging process.
• Clinical evaluation by fourteen physicians confirmed good structural integrity and standardized terminology in the generated reports.

Conclusions:

• The multimodal AI framework significantly streamlines SRUI workflows, making advanced microvascular imaging more accessible.
• This AI-assisted approach supports clinically contextualized super-resolution ultrasound imaging, potentially improving diagnostic efficiency and accuracy.
• The integration of LLMs and computer vision models offers a promising direction for the future of medical imaging analysis.