Comparison of Intern Doctors and ChatGPT in Emergency Cases Assessment

Accurate and timely diagnosis in emergency departments is crucial due to the high patient volume and time-sensitive nature of care. Intern doctors, who are nearing the completion of medical school, frequently work in emergency departments in many countries. However, after graduation, physicians are often expected to assume critical patient care responsibilities despite limited experience. Artificial intelligence models can quickly analyze patient data and generate diagnoses, thus assisting inexperienced physicians in enhancing diagnostic accuracy. This study aims to evaluate the diagnostic performance of ChatGPT-4 in emergency department case scenarios and compare its accuracy with that of intern doctors.

This study involved intern doctors participating in the internship program during the 2024-2025 academic year. A total of 36 case-based questions, categorized by difficulty level, were administered to 155 interns and subsequently presented to artificial intelligence. Descriptive statistics were used to summarize the data, and a one-sample t-test was conducted to compare the diagnostic accuracy between intern doctors and ChatGPT. Statistical significance was set at p<0.05.

Intern doctors achieved an overall correct response rate of 58.3%, while ChatGPT achieved a rate of 97.2%. A statistically significant, moderate negative correlation was found between question difficulty and interns' performance (r=-0.684; p<0.001), indicating decreased accuracy as question difficulty increased. ChatGPT consistently demonstrated significantly higher performance across all difficulty levels.

ChatGPT-4 may serve as a valuable diagnostic support tool in emergency departments, particularly for newly graduated physicians with limited clinical experience.