Related Experiment Video
Updated: Sep 7, 2026
Utilizing High Resolution Ultrasound to Monitor Tumor Onset and Growth in Genetically Engineered Pancreatic Cancer Models
Published on: April 7, 2018
Large language models for analyzing contrast-enhanced ultrasound reports of pancreatic cystic lesions
Yuming Shao1, Yang Gui1, Xiaoyi Yan1
1Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
Background:
Pancreatic cystic lesions (PCLs) require precise imaging characterization to guide clinical management. Contrast-enhanced ultrasound (CEUS) reports contain operator-dependent narratives that challenge clinicians. Large language models (LLMs) show potential in medical text analysis but lack validation for pancreatic CEUS interpretation. This study primarily aimed to evaluate the diagnostic accuracy of LLMs in interpreting Chinese CEUS reports of PCLs.
Methods:
We retrospectively analyzed 80 pathologically confirmed PCLs (21 benign, 25 borderline malignant, 34 malignant). Four LLMs (GPT-4o, Claude 3.7 Sonnet, Gemini 2.0, DeepSeek-R1) and eight radiologists (senior/junior =4:4) independently interpreted reports under three input modalities: grayscale-only (IM1), grayscale + CEUS (IM2), and demographics + grayscale + CEUS (IM3). A weighted scoring system (0-100 points per case, yielding a maximum total of 8,000 points) quantified alignment with pathology-defined categories (benign/borderline/malignant). LLM errors were categorized into four reasons. Junior radiologists reinterpreted cases with LLM assistance.
Results:
Under CEUS input conditions (IM2/IM3), LLMs showed no statistically significant difference from senior radiologists and significantly outperformed junior radiologists. The median score per case (out of 100), after averaging across the four LLMs, increased with input complexity (IM1: 47.50; IM2: 53.75; IM3: 73.75). Diagnostic accuracy varied by pathology: malignant lesions scored highest, while benign serous cystic neoplasms scored lowest due to suboptimal CEUS visualization and LLMs' knowledge gaps. LLM guidance elevated junior radiologists' accuracy to senior levels.
Conclusions:
LLMs show promising capability in interpreting CEUS reports of PCLs, with no statistically significant difference from senior radiologists in this dataset. Their integration significantly improves junior clinicians' interpretation of CEUS reports. These findings support further investigation of LLMs as potential auxiliary tools for ultrasound text analysis, though external validation is needed.