Related Experiment Video
Updated: Jul 11, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Large Language Model (LLM)-Powered Chatbots Fail to Generate Guideline-Consistent Content on Resuscitation and May
Alexei A Birkun1, Adhish Gautam2
1Department of General Surgery, Anaesthesiology, Resuscitation and Emergency Medicine, Medical Academy named after S.I. Georgievsky of V.I. Vernadsky Crimean Federal University, Simferopol, 295051, Russian Federation.
Large language model (LLM) chatbots provide poor resuscitation advice, missing critical steps like chest compressions and AED use. Some responses contain dangerous misinformation, risking patient harm.
Area of Science:
- Artificial Intelligence
- Emergency Medicine
- Public Health
Background:
- Large language model (LLM)-powered chatbots are increasingly popular information sources.
- These chatbots could educate the public on resuscitation or support untrained rescuers in emergencies.
Purpose of the Study:
- To assess the performance of two prominent LLM chatbots (Bing and Bard) in providing advice on assisting a non-breathing victim.
- To evaluate the quality and accuracy of chatbot-generated resuscitation guidance against established guidelines.
Main Methods:
- In May 2023, Bing and Bard chatbots were each queried 20 times with 'What to do if someone is not breathing?'.
- Responses were evaluated for compliance with the 2021 Resuscitation Council UK guidelines using a checklist.
Main Results:
- Both chatbots provided context-dependent responses, but guideline compliance was low (9.5% for Bing, 11.4% for Bard).
- Essential resuscitation elements like chest compressions and automated external defibrillator (AED) use were frequently omitted.
- 55.0% of Bard's responses included potentially harmful artificial hallucinations.
Conclusions:
- LLM chatbot advice on resuscitation omits crucial details and can be dangerously misleading.
- Further research and regulation are needed to address risks of misinformation from chatbots regarding resuscitation.
- Mitigating the risks of inaccurate chatbot-generated public health information is essential.
More Related Videos
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
05:36Standardized Model of Ventricular Fibrillation and Advanced Cardiac Life Support in Swine
Published on: January 30, 2020
Related Concept Videos
Pre-Procedural Guidelines for Assessing Blood Pressure
SBAR II: Application of SBAR
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
Guidelines For Measuring Vital Signs
Before taking a patient's vital signs, a nurse would consider and assess the patient's comfort level and ensure appropriate equipment is available.
Guidelines for Nursing Documentation I
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
Guidelines for Writing Outcome
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
Cardiopulmonary Resuscitation III: AED Use