Related Experiment Videos
Agreement and classification performance of ChatGPT-5-generated and nurse-led emergency department triage using an
Elif Bodur Tiryaki1, Oğuzhan Tiryaki2, Miray Soysal3
1Artvin Çoruh University, Health Services Vocational School, Artvin and Atatürk University Institute of Health Sciences, Child Health and Disease Nursing PhD, Erzurum, Turkey.
Abstract:
This study evaluated the agreement and classification performance of ChatGPT-5-generated triage compared with nurse triage using an expert reference decision in adult emergency department patients. In this retrospective cross-sectional study, 1564 adult patients with documented yellow or green triage classifications were included. Standardized triage-level patient information was entered into ChatGPT-5 using an identical prompt for all patients. Agreement was assessed using Cohen's κ, and classification performance was evaluated using sensitivity, specificity, and overall accuracy. The mean age was 39.8 ± 18.5 years, and 55.9% of patients were female. The expert reference decision classified 50.0% of patients as yellow, compared with 48.5% by ChatGPT-5-generated triage and 25.6% by nurse triage. ChatGPT-5-generated triage showed higher agreement with the expert reference decision than nurse triage (κ = 0.670 vs. 0.467), higher overall accuracy (83.5% vs. 73.3%), and higher sensitivity (82.0% vs. 49.0%). Compared with nurse triage, ChatGPT-5-generated triage produced fewer green classifications among expert-reference yellow patients (18.0% vs. 51.0%), whereas nurse triage produced fewer yellow classifications among expert-reference green patients (2.3% vs. 15.0%). These findings indicate that ChatGPT-5-generated triage more closely approximated the expert reference decision than nurse triage under the conditions evaluated in this study.