Related Experiment Video
Updated: Jun 12, 2026
Setting Up a Stroke Team Algorithm and Conducting Simulation-based Training in the Emergency Department - A Practical Guide
Published on: January 15, 2017
Comparison of the performance of ChatGPT Plus, Gemini Pro, and experienced nurses in pediatric emergency triage using
Pervin Erdem1, Aytaç Göktuğ2, Gülser Esen Besli2
1Pediatric Emergency Department, Nursing Division, Istanbul Medeniyet University Göztepe Prof. Dr. Süleyman Yalçın City Hospital, Istanbul, Turkey.
Aim:
We aimed to evaluate the performance of ChatGPT Plus and Gemini Pro, two large language model-based artificial intelligence (AI) tools, to predict actual triage levels of patients in the pediatric emergency department and compare them with the decisions of experienced nurses.
Design And Methods:
This single-center, prospective observational study was conducted between September 15 and October 15, 2025. Nurses received standardized refresher training on the Emergency Severity Index (ESI) before data collection. A triage nurse assessed each patient while a pediatric emergency medicine physician simultaneously determined the gold-standard triage level; AI triage levels were recorded by a separate researcher. Patients were classified as emergent (ESI-1 and 2), urgent (ESI-3), or non-urgent (ESI-4 and 5). Agreement was evaluated using the Quadratic Weighted Kappa (κ_w) coefficient.
Results:
A total of 378 patients were included. While near-perfect agreement was observed between the physician and nurses, agreement for ChatGPT Plus and Gemini Pro was moderate (κ_w 0.892, 0.586, and 0.544, respectively). The nurses had a higher triage accuracy than AI tools (92% compared to 65% for ChatGPT Plus and 63% for Gemini Pro, p < 0.001). AI tools showed a tendency toward under-triage of emergent patients and over-triage of both urgent and non-urgent patients (p = 0.038 and p < 0.001).
Conclusion:
These findings demonstrate that general AI tools are not yet sufficiently accurate to classify pediatric cases and cannot replace experienced nurses.
Implications To Practice:
Future hybrid AI models incorporating real-time clinical observations and age-specific physiological parameters may enhance applicability of these technologies in pediatric emergency triage.