The Application of Large Language Models for Radiologic Decision Making.

Area of Science:

• Artificial Intelligence in Medicine
• Radiology Decision Support

Background:

• Large language models (LLMs) are rapidly advancing.
• Their application in medical fields, particularly radiology, is underexplored.

Purpose of the Study:

• To evaluate the capability of LLMs in predicting optimal imaging studies for diverse clinical scenarios.
• To benchmark LLM performance against established radiology guidelines.

Main Methods:

• Two LLMs, ChatGPT and Glass AI, were tested on 1,075 clinical cases from 11 American College of Radiology (ACR) expert panels.
• Performance was measured against ACR Appropriateness Criteria using a 0-3 scoring scale, with results averaged for each panel.

Main Results:

• Glass AI (2.32 ± 0.67) significantly outperformed ChatGPT (2.08 ± 0.74).
• Both models performed best in Polytrauma, Breast, and Vascular imaging, and worst in Neurologic, Musculoskeletal, and Cardiac panels.
• Glass AI excelled in 10 out of 11 subspecialties.

Conclusions:

• LLMs show promise for aiding in radiologic decision-making.
• Specialized medical training, as seen in Glass AI, appears to improve LLM performance in predicting imaging studies.