评估大型语言模型,分析医疗保健事件报告中的安全风险
Kerstin Denecke1, Helmut Paula2
1Bern University of Applied Sciences, Bern, Switzerland.
Studies in health technology and informatics
|August 8, 2025
概括
大型语言模型 (LLM) 显示出分析医疗保健事件报告的前景. 虽然它们有效地提取事件和原因,但它们很难识别由于幻觉导致的促成因素.
科学领域:
- 医疗保健安全 医疗保健安全
- 自然语言处理自然语言处理.
- 人工智能在医学中的应用
背景情况:
- 事件报告对于识别医疗保健安全风险至关重要.
- 及时分析这些报告对于系统改进至关重要.
- 自然语言处理 (NLP) 提供了自动化报告分析的潜力.
研究的目的:
- 评估大型语言模型 (LLM) 在从医疗事件报告中提取信息的有效性.
- 评估LLM Gemma-2在识别事件事件,原因和促成因素方面的表现.
- 确定LLM在提高事件分析效率和一致性方面的潜力.
主要方法:
- 利用了来自瑞士国家关键事件数据库 (CIRRNET®) 的1063份事件报告.
- 应用了LLM Gemma-2来提取和主题事件,原因和促成因素.
- 手动评估了100份报告,以确定提取的信息的准确性.
主要成果:
- 杰玛-2在提取事件方面达到92%的准确率,在提取原因方面达到84%的准确率.
- 提取有助于因素的结果是72%的准确性,并指出了幻觉和误解的问题.
- 该LLM展示了提高事件报告分析速度和统一性的潜力.
结论:
- 像Gemma-2这样的LLM在改善医疗保健事件报告分析方面具有显著的潜力.
- 需要进一步发展,以解决提取细微信息的局限性,例如有助于因素.
- 应用LLM可以简化医疗保健系统中的安全风险分析.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
579
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.2K
相关概念视频
01:21Types of Reports II: Incident or Occurrence Report
951
An Incident or Occurrence Report in a healthcare setting is a crucial document used to record any unexpected occurrence that may or may not have affected a patient, employee, or visitor. Such reports are critical to improving patient safety and include all details leading up to and including the event.
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
951
01:26Types of Reports III: Telephone and Verbal Reports
807
Telephone and Verbal Reports in healthcare settings are two communication methods for conveying therapeutic instructions from healthcare providers to nurses or other healthcare staff.
Here's an overview of each type:
Telephone Orders
Here's an overview of each type:
Telephone Orders
807
01:18Steps in Outbreak Investigation
204
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
204
01:19Pharmacovigilance
993
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
993
01:15Methods of Documentation VI: Case Management Model
628
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
628
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
174
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
174