Related Experiment Video
Updated: May 9, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Can a Large Language Model Grounded in Text-Based Agency-Specific Prehospital Protocols Provide Accurate Care
Colin G Wang1, Nichole Bosson1,2,3, Rombod Rahimian4
1Department of Emergency Medicine, Harbor-UCLA Medical Center, Torrance, California.
Prehospital Emergency Care
|May 7, 2026
Summary
Retrieval-augmented generation (RAG) large language models (LLMs) achieved 75% accuracy in providing prehospital care recommendations based on emergency medical services (EMS) protocols. This study evaluated LLM accuracy for adult and pediatric emergency scenarios, finding no patient safety risks from identified hallucinations.
Area of Science:
- Artificial Intelligence in Healthcare
- Emergency Medical Services Research
- Clinical Decision Support Systems
Background:
- Large language models (LLMs) with retrieval-augmented generation (RAG) can provide source-grounded responses.
- Evaluating the accuracy of RAG-based LLMs for prehospital care recommendations is crucial for potential clinical integration.
Purpose of the Study:
- To assess the accuracy of a RAG-based LLM in generating prehospital care recommendations aligned with emergency medical services (EMS) policies and treatment protocols (TPs).
- To identify and categorize any inaccuracies or omissions in LLM-generated recommendations across diverse clinical scenarios.
Main Methods:
- A simulation-based study utilized Google's NotebookLM with a RAG-based LLM (Gemini 2.5 Flash).
- Text-based EMS policies/TPs from a large EMS system were uploaded.
- Six clinical scenarios (adult and pediatric) were developed, and LLM responses were independently evaluated for accuracy and hallucinations, categorizing missed actions by severity.
Main Results:
- The LLM provided 75% (127/169) of recommended patient care actions across all scenarios.
- 42 actions were missed, with 5% categorized as 'major misses' and 8% as 'minor misses'.
- Five major misses occurred in a pediatric out-of-hospital cardiac arrest scenario, primarily due to failure to prompt for secondary causes; 12 hallucinations were identified, none posing a safety risk.
Conclusions:
- A RAG-based LLM demonstrated 75% accuracy in generating prehospital care recommendations grounded in EMS policies and TPs.
- While generally accurate, specific areas like identifying secondary causes in pediatric emergencies require further refinement.
- The study highlights the potential of RAG-LLMs as decision support tools in prehospital care, with a need for continued validation and improvement.
Related Concept Videos
01:15Methods of Documentation VI: Case Management Model
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic illness...
For example, a patient with a chronic illness...
01:26Types of Reports III: Telephone and Verbal Reports
Telephone and Verbal Reports in healthcare settings are two communication methods for conveying therapeutic instructions from healthcare providers to nurses or other healthcare staff.
Here's an overview of each type:
Telephone Orders
Here's an overview of each type:
Telephone Orders
01:22Cardiopulmonary Resuscitation II: ACLS Airway Management
Airway management is a key skill in emergency and critical care settings, as maintaining a clear airway is essential for adequate oxygenation and ventilation.Head Tilt-Chin Lift TechniqueThe head tilt-chin lift maneuver is an essential technique primarily used in patients without suspected cervical spine injuries. To perform this maneuver, one hand is placed on the patient’s forehead, and gentle pressure is applied backward to tilt the head. The fingertips of the other hand are positioned under...
01:15Impact of Pharmacokinetic–Pharmacodynamic Models: Regulatory Decisions
PK–PD modeling has significantly influenced FDA regulatory decisions, particularly drug approval, dosage optimization, and labeling. These models integrate pharmacokinetics (PK) and pharmacodynamics (PD) to predict drug behavior and effects, aiding in optimizing dosing regimens and enhancing the probability of clinical trial success.One notable example is Nesiritide (Natrecor®), a recombinant human brain natriuretic peptide for treating acute decompensated congestive heart failure (CHF).
01:30Guidelines for Nursing Documentation I
Quality documentation and reporting share essential characteristics that ensure they are practical and valuable resources for those who use them. These characteristics are:
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
01:25Cardiopulmonary Resuscitation IV: Pharmacological Management
Pharmacologic intervention is crucial in treating cardiac arrest patients during ACLS or Advanced Cardiovascular Life Support. The ACLS algorithms guide the administration of specific drugs based on the patient's cardiac arrest rhythm, which includes pulseless ventricular tachycardia (VT), ventricular fibrillation (VF), asystole, and pulseless electrical activity (PEA).EpinephrineIndication: Epinephrine is the first-line drug for all cardiac arrest rhythms.Mechanism of Action: Epinephrine...