Improving Trauma Triage Accuracy with Large Language Models: A Comparison to Human Expert Decisions

Accurate prehospital trauma triage and communication determine morbidity, mortality, and system efficiency. Advancements in large language models (LLMs) offer an opportunity to improve triage and yet to be implemented in prehospital trauma triage.

This retrospective cohort study evaluates LLM performance in trauma triage and accuracy of prehospital tele-communication. Of 410 pediatric activations at a Level I center (January 2023 to May 2025, IRB no. 00009569), 133 activations with emergency medical service recordings, human-generated trauma pages, and injury severity scores were analyzed. Audio was transcribed with OpenAI Whisper. Structured "Essential Transcripts" were generated with named entity recognition. Entity ablation tested redaction of triage parameters on accuracy. In a prospective arm, trauma surgeons reviewed emergency medical service transcripts and triaged activations pre- and post-LLM exposure. Cribari criteria defined over and undertriage. McNemar's test and 95% CIs assessed paired differences in accuracy.

The primary endpoint: LLM undertriage demonstrated modest improvement; 4.8% (3 to 8.2) vs 5.1% (3.1 to 9.3, p = 0.73, Bonferroni p = 1). For secondary endpoints, LLM triage outperformed human clinicians: 83.5% accuracy (80.6 to 90.6) vs 78.9% (73.9 to 82.6, p < 0.01, Bonferroni p < 0.01), with overtriage 58.6% (51.4 to 73.7) vs 71.8% (p < 0.05, Bonferroni p < 0.09). "Essential Transcripts" reduced transcript length by 80.8% (p < 0.001) while preserving accuracy (81.9%, 76.6 to 87.5, p < 0.001, Bonferroni p < 0.05). Entity ablation had marginal effect on triage. In prospective evaluation, human triage accuracy improved after LLM exposure (73.7% [69.8 to 77.2]) to 75.8% ([71.9 to 79.2], p = 0.04, Bonferroni p = 0.12), significantly improving the odds of a correct triage decision (odds ratio 2.57, 95% CI 1.39 to 6.83, p < 0.01).

LLMs achieve triage accuracy comparable to trauma staff in retrospective review of pediatric trauma. Further validation is needed to assess clinical outcomes, generalizability, and user acceptance before widespread deployment.