Related Experiment Video
Updated: Apr 22, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Large language model-assisted radiology reporting in a single-radiologist implementation: a retrospective cohort
1Department of Radiology, Mayo Clinic, Phoenix, USA. tan.nelly@mayo.edu.
Background:
Radiologist burnout affects approximately 40% of US radiologists. Large language models (LLMs) may improve workflow efficiency, but real-world implementation data are limited.
Objective:
To evaluate the impact of an LLM-assisted workflow on radiologist efficiency using the Unified Theory of Acceptance and Use of Technology (UTAUT) framework.
Design, Setting, And Participants:
HIPAA-compliant, IRB-approved retrospective cohort study of a single fellowship-trained abdominal radiologist exploratory study at Mayo Clinic Arizona. We compared baseline (January-April 2024) and post-implementation (December 2025-February 2026) periods. A custom generative pre-trained transformer was developed using ChatGPT Enterprise Model 5.2 Thinking with disease-specific templates.
Main Outcome Measures:
Inter-study interval time, used as a proxy for interpretation time, compared using Wilcoxon rank-sum tests with Bonferroni correction (α = 0.01). UTAUT constructs assessed: performance expectancy (efficiency), effort expectancy (training burden), facilitating conditions (infrastructure), and behavioral intention (satisfaction).
Results:
We analyzed 609 studies (495 CT, 114 MRI). LLM assistance significantly reduced inter-study intervals for outpatient CT with contrast (23.0 vs. 13.0 min; difference 10 min; p = 0.0021) and without contrast (18.5 vs. 7.0 min; difference 11.5 min; p = 0.0017). No improvement occurred for MRI with contrast (14.0 vs. 16.0 min; p = 0.2808) or without contrast (14.0 vs. 7.0 min; p = 0.0889). The radiologist reported improved work-life balance for CT but neutral satisfaction for complex MRI templates. Training required 10 h over 5 days.
Conclusions:
LLM-assisted workflow reduced inter-study interpretation times for standardized CT studies and no clear efficiency benefit was observed for MRI in this small implementation sample, when interpreted through a UTAUT lens, particularly on performance expectancy and task-technology fit as adoption drivers. Efficiency gains may reduce documentation burden when tools align with task complexity.
