Related Experiment Videos
Automated Quality Control of Breast Ultrasound Reports Using a BI-RADS-Prompted Large Language Model: A Pilot
Siyu Lu1, Hongyan Wang2, Jian Wang3
1Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China; National Ultrasound Medical Quality Control Center, Beijing, China.
Journal of the American College of Radiology : JACR
|July 22, 2026
Summary
Large language models (LLMs) show promise for automated quality control (QC) of breast ultrasound (US) reports, achieving accuracy comparable to human experts but with significantly greater efficiency. This AI-driven approach enhances the reliability of breast US diagnostic reporting.
Area of Science:
- Radiology and Medical Imaging
- Artificial Intelligence in Healthcare
- Medical Informatics
Background:
- Quality control (QC) of breast ultrasound (US) reports is crucial for accurate diagnosis.
- Manual QC processes can be time-consuming and prone to variability.
- Automating QC could improve efficiency and consistency in reporting.
Purpose of the Study:
- To assess the feasibility and performance of a large language model (LLM) for automated QC of breast US reports.
- To compare the accuracy and efficiency of LLM-based QC against manual QC by personnel.
Main Methods:
- A retrospective multicenter study involving 735 breast US reports from 60 hospitals.
- Free-text reports were standardized by QC personnel and an LLM (Qwen2.5-VL-7B).
- A gold standard was established via expert review; LLM and manual QC accuracy/efficiency were compared.
Main Results:
- The LLM achieved higher accuracy in QC for key features like margin (80.0% vs 64.1%) and echo pattern (74.0% vs 56.1%).
- LLM demonstrated consistent performance in complex reports and a positive correlation with BI-RADS categories (3-5).
- LLM completed QC for 50 reports in 13 minutes, significantly faster than manual reviewers (212.5 minutes).
Conclusions:
- LLM-based systems offer a reliable, accurate, and efficient solution for breast US report QC.
- The LLM achieves human-comparable performance with markedly higher efficiency.
- This technology can enhance breast US report quality, especially for high-suspicion cases.