Improving Trauma Triage Accuracy with Large Language Models: A Comparison to Human Expert Decisions

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Trauma Surgery

Background:

• Accurate prehospital trauma triage is critical for patient outcomes and healthcare system efficiency.
• Large language models (LLMs) present a novel opportunity to enhance prehospital trauma triage processes.
• Current implementation of LLMs in prehospital trauma care is limited.

Purpose of the Study:

• To evaluate the performance of LLMs in pediatric trauma triage.
• To assess the accuracy of LLM-assisted prehospital tele-communication.
• To compare LLM triage accuracy against human clinician performance.

Main Methods:

• Retrospective cohort study of 133 pediatric trauma activations at a Level I center.
• Analysis of EMS recordings, trauma pages, and Injury Severity Scores (ISS).
• Utilized OpenAI Whisper for transcription and Named Entity Recognition (NER) for structured data extraction; prospective evaluation involved trauma surgeons.

Main Results:

• LLM triage demonstrated comparable accuracy to human clinicians in retrospective analysis (83.5% vs. 78.9%).
• LLM-assisted triage showed a trend towards reduced under-triage (4.8% vs. 5.1%) and significantly reduced over-triage (58.6% vs. 71.8%).
• Prospective evaluation indicated improved human triage accuracy after LLM exposure, enhancing correct triage decision odds.

Conclusions:

• LLMs achieve comparable accuracy to trauma staff in retrospective pediatric trauma triage.
• The use of structured "Essential Transcripts" significantly reduced data length while maintaining accuracy.
• Further research is essential to validate LLM generalizability, clinical outcomes, and user acceptance for widespread deployment.