一个尺寸适合所有吗? 制定评估策略,以评估患者安全事件报告分析的大型语言模型
Allan Fong1, Katharine T Adams1, Christian Boxley1
1Center for Biostatistics, Informatics, and Data Science, MedStar Health Research Institute, MedStar Health, Washington, DC 20008, United States.
对患者安全事件 (PSE) 报告分析进行大型语言模型 (LLM) 的评估至关重要. 不同的LLM在临床或一般语言方面表现出色,这表明需要量身定制的方法来有效分析PSE报告.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 患者安全 患者安全
背景情况:
- 分析患者安全事件 (PSE) 报告对于改善医疗保健安全至关重要.
- 传统的分析方法面临着PSE报告语言的混合性质的挑战.
- 大型语言模型 (LLM) 提供了增强PSE报告分析的潜力.
研究的目的:
- 开发和应用一个数据驱动的策略来评估LLMs的适合性,以分析PSE报告.
- 使用特定指标评估不同LLM的理解能力.
主要方法:
- 提出了一种新的评估策略,重点关注PSE报告中的目标令牌和句子.
- 困惑度被用作一个关键的指标来定量评估LLM的表现.
- 根据他们对PSE报告内容的理解,评估了四个不同的LLM.
主要成果:
- 在评估的LLM中,在七个事件类别中的六个中观察到困惑度得分的显著差异.
- 临床LLM在临床叙述方面表现出卓越的表现,这些叙述通常出现在护士和医生报告中.
- 一般目的的LLM在通俗语言和以沟通为中心的主题报告中表现更好.
结论:
- 在PSE报告分析中LLM的有效性取决于模型的培训数据与报告的语言特征之间的协调.
- 一个适合所有LLM方法可能不是最佳的;专业或混合策略可能更有利于全面的PSE报告分析.
更多相关视频
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
相关概念视频
Types of Reports III: Telephone and Verbal Reports
Here's an overview of each type:
Telephone Orders
Types of Reports II: Incident or Occurrence Report
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
Actuarial Approach
Consider the example of a high-risk surgical procedure with significant early-stage mortality. A two-year clinical study is conducted,...
Assumptions of Survival Analysis
