The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology Workflow Optimization
• Large Language Models (LLMs)

Background:

• The application of large language models (LLMs) in radiology is an emerging field.
• The impact of LLMs on radiologists' diagnostic workflow and accuracy is not well-understood.
• GPT-4 represents a state-of-the-art LLM with potential applications in healthcare.

Purpose of the Study:

• To investigate the effect of GPT-4 assistance on radiologists' diagnostic accuracy and confidence.
• To evaluate the impact of AI on the overall diagnostic workflow.
• To identify potential risks associated with AI assistance in radiology.

Main Methods:

• A retrospective study involving six radiologists of varying experience levels.
• Radiologists interpreted 40 imaging studies (radiography, CT, MRI, angiography) with and without GPT-4 assistance.
• Diagnostic accuracy, confidence, user experience, and AI response quality were assessed using an online survey and statistical analysis.

Main Results:

• AI assistance led to a slight improvement in diagnostic accuracy (75.4% to 78.3%).
• Radiologists reported significantly higher diagnostic confidence when using GPT-4 (p<0.001).
• GPT-4 provided incorrect information (hallucinations or misinterpretations) in 7.4% of responses, necessitating caution.

Conclusions:

• Integrating GPT-4 into the diagnostic process can enhance radiologist confidence and slightly improve accuracy.
• The presence of AI-generated errors underscores the need for robust validation and safeguarding measures.
• Further research is required to optimize the safe and effective use of LLMs in clinical radiology.