Related Experiment Video
Updated: Aug 14, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Utility of large language models in pediatric trauma triage: An age-stratified analysis of prompt engineering
Brendan T Fox1, Ascharya Kushidhan Balaji1, Philip Seger1
1Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, NY, USA.
Insights
Structured prompt engineering significantly enhances large language models (LLMs) for pediatric trauma triage, matching clinician performance across all age groups. Prompt design, not the LLM itself, is key to improving decision support.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Medicine
- Pediatric Emergency Medicine
Background:
- Pediatric trauma triage faces challenges due to age-related physiological differences and infrequent cases, leading to inconsistent outcomes.
- Standardized guidelines often result in undertriage and overtriage, necessitating improved decision-support tools.
- Large language models (LLMs) show promise for aiding pediatric trauma triage across diverse age groups.
Purpose of the Study:
- To evaluate the effectiveness of different large language model (LLM) prompting strategies for pediatric trauma triage.
- To compare LLM performance against clinician decisions and a validated injury severity standard.
- To determine the impact of prompt design and LLM selection on triage accuracy.
Main Methods:
- A retrospective analysis of 354 pediatric trauma audio recordings from two Level I trauma centers was conducted.
- Audio recordings were transcribed and processed using four LLM prompting strategies (Simple Zero-Shot, L1-biased, L2-biased, Ensemble) and three input formats.
- LLM performance was assessed against clinician triage and Injury Severity Score (ISS ≥16), evaluating accuracy, sensitivity, specificity, undertriage, overtriage, and agreement (Cohen's κ).
Main Results:
- Prompt engineering substantially improved LLM accuracy, with the Ensemble strategy achieving 83.9% accuracy, surpassing clinicians (78.8%).
- The Ensemble prompt strategy demonstrated equivalent undertriage (31.9% vs. 34.0%) and reduced overtriage (13.7% vs. 19.2%) compared to clinicians.
- Prompt strategy had the most significant impact on accuracy (Δ21.5%), while LLM selection and input format had minimal effects; performance was consistent across pediatric age groups.
Conclusions:
- Structured prompt engineering allows general-domain LLMs to achieve performance comparable to clinicians in pediatric trauma triage.
- The design of prompts, rather than the choice of LLM, is the primary factor influencing performance.
- These findings highlight the potential of structured prompt engineering to enhance the utility of off-the-shelf LLMs for pediatric triage decision support.
Background:
Pediatric trauma triage is complicated by age-dependent physiologic variability and low case volumes, contributing to persistent undertriage and overtriage despite standardized guidelines. Large language models (LLMs) may offer potential decision-support regardless of pediatric age group.
Methods:
We performed a retrospective analysis of 354 EMS-to-hospital pediatric trauma audio recordings from two Level I pediatric trauma centers (2023-2025). Audio recordings were transcribed and evaluated across four LLM prompting strategies (Simple Zero-Shot, L1-biased, L2-biased, Ensemble), three input formats (raw, processed, structured), and three LLMs. Performance was assessed relative to clinician triage decisions and an Injury Severity Score (ISS ≥16) reference standard. Outcomes included accuracy, sensitivity, specificity, undertriage, overtriage, and agreement (Cohen's κ).
Results:
Prompt engineering significantly improved LLM performance. Compared with Simple Zero-Shot prompting (71.5% accuracy), criteria-based strategies achieved higher accuracy, with the Ensemble approach demonstrating the most balanced performance (83.9% vs clinicians 78.8%; p = 0.026), equivalent undertriage (31.9% vs 34.0%; p = 1.0), and lower overtriage (13.7% vs 19.2%; p = 0.022). Performance remained stable across age groups (all p > 0.10). Prompt strategy exerted the largest effect on accuracy (Δ21.5%), while model selection (Δ0.7%) and input format (Δ2.2%) had minimal impact. Prompt strategies produced distinct tradeoffs between sensitivity and specificity.
Conclusions:
Structured prompt engineering enables general-domain LLMs to achieve clinician-comparable performance in pediatric trauma triage across age groups. Prompt design, rather than model selection, was the primary determinant of performance. These findings support that structured prompt engineering improves off-the-shelf LLM performance over unstructured prompting across age groups.