Related Experiment Video
Updated: Jun 22, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
543
Emergency Patient Triage Improvement through a Retrieval-Augmented Generation Enhanced Large-Scale Language Model
Megumi Yazaki1,2,3, Satoshi Maki1,4, Takeo Furuya1
1Department of Orthopaedic Surgery, Graduate School of Medicine, Chiba University, Chiba, Japan.
Prehospital Emergency Care
|July 1, 2024
Summary
Retrieval Augmented Generation (RAG) with Large Language Models (LLMs) significantly improved emergency medical triage accuracy. This AI integration reduced under-triage rates, showing promise for standardizing emergency care.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
- Emergency Medicine
Background:
- Emergency medical triage is critical for patient care but can be inconsistent.
- Variability in triage effectiveness is often linked to personnel experience and training.
Purpose of the Study:
- To evaluate the efficacy of integrating Retrieval Augmented Generation (RAG) with Large Language Models (LLMs) for standardizing emergency medical triage.
- To reduce variability in emergency care through AI-assisted triage procedures.
Main Methods:
- Developed 100 simulated triage scenarios based on Japanese National Examination for Emergency Medical Technicians cases.
- Processed scenarios using RAG-enhanced LLMs (GPT-3.5), inputting patient vital signs, symptoms, and EMS observations.
- Compared LLM triage accuracy against Emergency Medical Technicians (EMTs) and emergency physicians, measuring primary outcome (accuracy) and secondary outcomes (under-triage, over-triage rates).
Main Results:
- The RAG-enhanced GPT-3.5 model achieved a 70% correct triage rate, significantly outperforming EMTs (35-38%) and physicians (47-50%).
- The RAG-GPT-3.5 model reduced under-triage rates to 8%, a substantial improvement compared to GPT-3.5 without RAG (33%) and GPT-4 without RAG (39%).
Conclusions:
- Integrating RAG with LLMs demonstrates potential for enhancing accuracy and consistency in emergency medical assessments.
- Further validation in diverse clinical settings with extensive datasets is required to confirm the real-world effectiveness and adaptability of these AI technologies.
