ChatGPT: A Useful Tool for Medical Students in Radiology Education?

Area of Science:

• Medical Education
• Artificial Intelligence in Healthcare
• Radiology

Background:

• Large language models (LLMs) are being explored as educational tools in medical training, especially in radiology.
• This study assesses ChatGPT's accuracy in recommending imaging investigations for various clinical scenarios.

Purpose of the Study:

• To evaluate ChatGPT's accuracy in recommending appropriate imaging investigations.
• To compare ChatGPT's performance with that of medical students and junior doctors.
• To explore the potential of LLMs as educational tools in radiology.

Main Methods:

• ChatGPT-4 (March 2024) was given a 12-case questionnaire based on the American College of Radiology's Appropriateness Criteria (ACR-AC).
• ChatGPT's recommendations were compared against ACR-AC guidelines and the performance of 160 final-year medical students and junior doctors.

Main Results:

• ChatGPT achieved 100% concordance with expert panel recommendations (12/12 cases).
• The student/doctor cohort achieved a 68.0% concordance rate.
• The difference in performance was statistically significant (p < 0.05).

Conclusions:

• ChatGPT demonstrated high accuracy in recommending appropriate imaging investigations based on guidelines.
• LLMs show potential as valuable adjuncts in radiology education, aiding less experienced clinicians.
• Further validation in real-world clinical settings is necessary.