ChatGPT: A Useful Tool for Medical Students in Radiology Education?

Large language models (LLMs) such as ChatGPT are increasingly being explored as educational tools in medical education, particularly in radiology. This study evaluated the accuracy of ChatGPT in recommending appropriate imaging investigations across diverse clinical scenarios, with a focus on its potential as an educational tool for medical students and junior doctors.

ChatGPT-4 (March 2024 version) was presented with a 12-case questionnaire derived from the American College of Radiology's Appropriateness Criteria (ACR-AC). One topic was selected from each of 10 diagnostic sections and two from the interventional section. The model's recommendations were compared with those published by the ACR-AC, which are based on expert consensus. The same questionnaire was also completed by 160 final-year medical students and junior doctors, and their collective performance was compared to ChatGPT.

ChatGPT achieved a 100% concordance rate (12/12 scenarios) with expert panel recommendations. In contrast, the student/doctor cohort achieved a 68.0% concordance rate. The difference was statistically significant (p < 0.05).

ChatGPT demonstrated high accuracy in recommending appropriate imaging investigations in a structured, guideline-based setting. These findings suggest that LLMs may serve as a valuable adjunct in radiology education, particularly in supporting imaging decision making among less experienced clinicians. However, further validation in real-world clinical environments is warranted.