Evaluating Large Language Models for Analysing Safety Risks in Healthcare Incident Reports
These videos have been matched automatically. Contact us if they are not relevant.
These videos have been matched automatically. Contact us if they are not relevant.
View abstract on PubMed
Summary
This summary is machine-generated.Large language models (LLMs) show promise for analyzing healthcare incident reports. While effective at extracting events and causes, they struggle with identifying contributing factors due to hallucinations.
Area Of Science
- Healthcare Safety
- Natural Language Processing
- Artificial Intelligence in Medicine
Background
- Incident reports are crucial for identifying healthcare safety risks.
- Timely analysis of these reports is essential for system improvement.
- Natural Language Processing (NLP) offers potential for automating report analysis.
Purpose Of The Study
- To evaluate the efficacy of Large Language Models (LLMs) in extracting information from healthcare incident reports.
- To assess the performance of the LLM Gemma-2 in identifying incident events, causes, and contributing factors.
- To determine the potential of LLMs to enhance the efficiency and consistency of incident analysis.
Main Methods
- Utilized 10,063 incident reports from the Swiss national critical incident database (CIRRNET®).
- Applied the LLM Gemma-2 to extract and theme events, causes, and contributing factors.
- Manually assessed 100 reports for the accuracy of extracted information.
Main Results
- Gemma-2 achieved 92% accuracy in extracting events and 84% accuracy in extracting causes.
- Extraction of contributing factors yielded 72% accuracy, with noted issues of hallucination and misinterpretation.
- The LLM demonstrated potential for improving the speed and uniformity of incident report analysis.
Conclusions
- LLMs, like Gemma-2, offer significant potential for improving the analysis of healthcare incident reports.
- Further development is needed to address limitations in extracting nuanced information such as contributing factors.
- The application of LLMs can streamline safety risk analysis in healthcare systems.
These videos have been matched automatically. Contact us if they are not relevant.
These videos have been matched automatically. Contact us if they are not relevant.
Related Concept Videos
An Incident or Occurrence Report in a healthcare setting is a crucial document used to record any unexpected occurrence that may or may not have affected a patient, employee, or visitor. Such reports are critical to improving patient safety and include all details leading up to and including the event.
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
Telephone and Verbal Reports in healthcare settings are two communication methods for conveying therapeutic instructions from healthcare providers to nurses or other healthcare staff.
Here's an overview of each type:
Telephone Orders
Usage: Telephone orders are used when a healthcare provider needs to communicate therapeutic instructions to a nurse or other staff over the phone. It often happens when the provider cannot be physically present.
Confirmation and Documentation: Oral...
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
Predicting Outbreaks
Predictive analytics, a branch of statistics, uses historical data, algorithmic models, and...
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...