Related Experiment Video
Updated: Jan 10, 2026
Author Spotlight: Improving Radiation Therapy Access with Radiation Planning Assistant
Published on: October 6, 2023
Advancing AI in radiology: a comparative analysis of ChatGPT-o1-preview and ChatGPT-3.5 in the Polish National
Adam Mitręga1,2, Michał Bielówka1,2, Dominika Kaczyńska1
1Students' Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine of the Medical University of Silesia in Katowice, Poland.
Purpose:
The aim of this study was to evaluate the performance of the ChatGPT-o1-preview language model in solving the Polish National Specialization Exam (PES) in radiology and imaging diagnostics and compare its results with previous versions of the model.
Material And Methods:
A set of 119 valid radiology exam questions from Spring 2023 was analyzed. Each question was classified by type, subtype, and clinical relevance. ChatGPT answered each question five times using a standardized prompt with a 5-point confidence scale. Performance was assessed using accuracy and declared and calculated difficulty indices. Statistical analysis was performed in Python with a significance level of p < 0.05, and results were compared with a previous model version.
Results:
The model achieved a score of 93.33% correct answers, comparable to the average physician score of 94.86%. ChatGPT-o1-preview showed exceptional accuracy in "memory" questions, with over 96% correct answers. This result, significantly higher than that of the older ChatGPT-3.5 model (52%), demonstrates progress in artificial intelligence (AI) capabilities. The model also exhibited higher confidence in its responses, indicating better adaptation to medical exams.
Conclusions:
Despite its high accuracy, the study was based on a relatively small set of questions, which limits the ability to fully assess the model's effectiveness. The results indicate the potential of AI as a tool to support clinical work, but further, more extensive research is necessary to evaluate its applicability and reliability in the medical environment.
More Related Videos
07:45Author Spotlight: Standardizing Mouse In Vivo PET Imaging with Body Conforming Molds and Automated Analysis
Published on: October 25, 2024
10:28Gene Regulation and Targeted Therapy in Gastric Cancer Peritoneal Metastasis: Radiological Findings from Dual Energy CT and PET/CT
Published on: January 22, 2018
Related Concept Videos
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
01:29Positron Emission Tomography
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
01:30Radiological Investigation I: X-ray and CT
01:27Imaging Studies III: Computed Tomography
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
01:29Pulmonary Embolism II: Diagnostic Studies and Interprofessional Care