MedFactEval和MedAgentBrief:一个框架和工作流程,用于生成和评估事实临床总结
Françcois Grolleau1, Emily Alsentzer2, Timothy Keyes3
1Center for Biomedical Informatics Research, Stanford University, Stanford, CA, USA, grolleau@stanford.edu.
Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing
|February 27, 2026
概括
从大型语言模型 (LLM) 中评估临床文本是一项挑战. 一个新的框架,MedFactEval,使用LLM评审团进行可扩展的,基于事实的评估,与人类专家达成高度一致.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 临床自然语言处理 临床自然语言处理
背景情况:
- 评估由大型语言模型 (LLM) 生成的临床文本中的事实准确性对于采用至关重要.
- 手动专家评审对于LLM生成的临床内容的持续质量保证是不可扩展的.
研究的目的:
- 引入MedFactEval,这是一个可扩展的框架,用于基于事实的评估LLM生成的临床文本.
- 介绍MedAgentBrief,这是一个工作流程,用于生成高质量的,事实性的临床出院摘要.
- 通过人类专家的共识建立的黄金标准来验证MedFactEval框架.
主要方法:
- MedFactEval使用临床医生定义的关键事实和"LLM陪审团" (多LLM多数投票) 来进行评估.
- MedAgentBrief是一个模型不可知,多步骤的工作流程,用于生成放电总结.
- 在住院病例的关键事实上,使用七名医生的多数投票创建了一个黄金标准.
主要成果:
- MedFactEval LLM评审团与人类专家小组几乎完全一致 (科恩的 κ = 81%).
- 在统计学上,LLM评审团的表现与单一的人类专家的表现不逊色 (κ = 67%,P < 0.001).
结论:
- MedFactEval提供了一个强大的和可扩展的框架,用于评估临床LLM输出中的事实准确性.
- MedAgentBrief为事实临床总结提供了一个高性能生成工作流.
- 结合的方法有助于在临床环境中负责任地部署生成性AI.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.7K
09:20Cloud-Based Phrase Mining and Analysis of User-Defined Phrase-Category Association in Biomedical Publications
Published on: February 23, 2019
9.3K
相关概念视频
01:17Flow Sheet
2.9K
Flowsheets are valuable tools in nursing documentation. They enable healthcare professionals to efficiently record and monitor various patient assessments and measurements in a consolidated format.
Here's a closer look at the examples of flowsheets commonly used by nurses:
Graphic Sheet Documentation:
Here's a closer look at the examples of flowsheets commonly used by nurses:
Graphic Sheet Documentation:
2.9K
01:11Clinical Trials: Overview
5.1K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
5.1K
01:28Formats for Nursing Documentation
2.1K
Nursing documentation encompasses various formats designed to capture precise patient data, facilitate communication among healthcare team members, and ensure comprehensive and accurate patient records. Let's explore each of these formats in detail:
Nursing Assessment Form:
• A nursing assessment form is a foundational document that captures detailed patient data from physical assessments and nursing histories.
• It includes patient demographics, medical history,...
Nursing Assessment Form:
• A nursing assessment form is a foundational document that captures detailed patient data from physical assessments and nursing histories.
• It includes patient demographics, medical history,...
2.1K
01:31Discharge Summary Forms
1.3K
The discharge summary is crucial as it enables a smooth transition from a healthcare facility to a patient's home or another care setting. This critical document facilitates seamless continuity of care, ensuring patients receive the necessary support and attention.
Here's a detailed look at the key components and guidelines for preparing a discharge summary:
Here's a detailed look at the key components and guidelines for preparing a discharge summary:
1.3K
01:27Nursing Clinical Information System
1.3K
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
1.3K
01:18Methods of Documentation I: Source-Oriented Records
1.8K
Source-oriented records, or SOR, are medical record-keeping organized by the data source. The SOR system was first developed in the mid-1900s to organize the growing patient data in hospitals and other healthcare facilities.
In an SOR, each discipline involved in patient care maintains a separate medical record section. This record-keeping method enables easy tracking of patient progress and ensures healthcare staff have access to up-to-date information.
Key Attributes include the following:
In an SOR, each discipline involved in patient care maintains a separate medical record section. This record-keeping method enables easy tracking of patient progress and ensures healthcare staff have access to up-to-date information.
Key Attributes include the following:
1.8K