Potential of GPT-4 for Detecting Errors in Radiology Reports: Implications for Reporting Accuracy

Area of Science:

• Medical Imaging and Diagnostics
• Artificial Intelligence in Healthcare
• Radiology Report Quality Assurance

Background:

• Radiology reports are prone to errors stemming from resident-attending discrepancies, speech recognition issues, and high workloads.
• Large language models (LLMs) present a potential solution for enhancing report generation and accuracy.

Purpose of the Study:

• To evaluate the effectiveness of GPT-4 in identifying common errors within radiology reports.
• Focus areas included performance, time efficiency, and cost-effectiveness of GPT-4's error detection capabilities.

Main Methods:

• A retrospective study analyzed 200 radiology reports (radiography, CT, MRI) with intentionally inserted errors.
• GPT-4 and six radiologists (senior, attending, resident) independently detected errors across five categories.
• Error detection rates, performance across categories, and reading times were statistically analyzed.

Main Results:

• GPT-4 achieved an 82.7% error detection rate, comparable to the average performance of radiologists across experience levels.
• One senior radiologist demonstrated a higher detection rate (94.7%) than GPT-4.
• GPT-4 significantly reduced processing time (3.5 seconds vs. 25.1 seconds) and correction costs ($0.03 vs. $0.42) per report compared to the fastest human reader.

Conclusions:

• GPT-4 demonstrates comparable error detection performance to radiologists in radiology reports.
• The integration of GPT-4 holds potential for reducing radiologist work hours and associated costs.
• LLMs offer a promising avenue for improving the quality and efficiency of radiological reporting.