Scenario-based comparative evaluation of ChatGPT-4o and physician groups in pediatric minor head trauma

Interest in the use of ChatGPT-4o in scenario-based clinical assessment has increased substantially in recent years. However, studies evaluating ChatGPT-4o in pediatric head trauma scenarios and comparing it with different physician groups remain limited.

To evaluate the scenario-based performance of ChatGPT-4o in pediatric head trauma and compare it with that of emergency physicians, neurosurgeons, and pediatricians.

This study included 60 pediatric patients who presented between 15 December 2024 and 15 June 2025 and met the inclusion criteria. After clinical follow-up, cases were converted into multiple-choice case scenarios and classified into red, yellow, and green zones according to PECARN. These scenarios were answered by 42 physicians from emergency medicine, neurosurgery, and pediatrics (n=14 per group) and by ChatGPT-4o. Concordance of scenario-based management responses with PECARN recommendations was compared statistically.

Of the 60 cases, 25.0% (n=15) were classified as red zone, 50.0% (n=30) as yellow zone, and 25.0% (n=15) as green zone. ChatGPT-4o showed lower scenario-based performance than all physician groups in red-zone cases. When non-contrast brain CT was accepted as the correct option in the yellow zone, ChatGPT-4o had the lowest overall accuracy (median: 24.50). When observation was accepted as correct, ChatGPT-4o showed the highest accuracy both in the yellow zone (median: 17.00; p=0.001) and overall (median: 35.50; p<0.001). ChatGPT-4o showed the highest accuracy in green-zone cases (median: 8.50).

ChatGPT-4o did not demonstrate adequate scenario-based performance in critical pediatric head trauma cases. However, it may have potential as a supportive tool in non-critical case scenarios.