开发和评估大型语言模型生成的紧急医疗交付笔记
Vince Hartman1, Xinyuan Zhang1, Ritika Poddar1
1Abstractive Health, New York, New York.
JAMA network open
|December 3, 2024
概括
大型语言模型 (LLM) 生成的紧急医疗 (EM) 交付笔记显示出优越的自动化评估指标,但比医生写的笔记少得一点有用和安全. 建议在医生循环中实施,以便在医疗保健中实现最佳的LLM整合.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床文档 临床文档
背景情况:
- 在紧急医疗 (EM) 中,医生的文档负担很大.
- 从EM转移到住院 (IP) 设置需要准确和安全的信息传输.
- 大型语言模型 (LLM) 提供了自动化临床文档的潜力.
研究的目的:
- 开发和评估LLM生成的EM-to-IP交付笔记. 为了开发和评估LLM生成的EM-to-IP交付笔记.
- 将LLM生成的笔记与医生撰写的笔记的准确性和患者安全性进行比较.
主要方法:
- 来自纽约长老会/威尔康奈尔医学中心 (1600名EM患者记录) 的队列研究 (2023年).
- 开发了一个定制的临床LLM管道,以生成模板化的EM-to-IP交付笔记.
- 评估使用了自动化指标 (ROUGE,BERTScore,SCALE) 和一个新的患者安全框架,包括医生审查.
主要成果:
- 与医生写的笔记相比,LLM生成的笔记在词汇相似性 (ROUGE,BERTScore) 和忠实性 (SCALE) 上显示出更高的分数.
- 一个亚样本的医生审查表明,LLM笔记的有用性略低 (4.04/5与4.36/5) 和安全性略低 (4.06/5与4.50/5).
- 在LLM生成的摘要中没有发现关键患者安全风险.
结论:
- 通过LLM生成的EM-to-IP交付笔记显示出希望,在自动化指标上表现优于医生笔记.
- 实用性和安全性的轻微劣势表明,需要在LLM实施中采用医生在循环中的方法.
- 该研究为测量LLM临床工具的实施前患者安全提供了一个框架.
更多相关视频
09:52Setting Up a Stroke Team Algorithm and Conducting Simulation-based Training in the Emergency Department - A Practical Guide
Published on: January 15, 2017
17.1K
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
15.8K
相关概念视频
Types of Reports I: Hands-off Report
886
A hand-off report, also known as a change-of-shift report, is a crucial nursing process that ensures the smooth transition of patient care responsibilities between nursing staff.
Following are the key components and categories of hand-off reports:
Purpose and Process:
Following are the key components and categories of hand-off reports:
Purpose and Process:
886
SBAR I: Understanding the Concept
4.4K
Effective communication among healthcare professionals during hand-off reporting is essential to delivering safe and continuous patient care. Common professional interactions include reports to healthcare team members, hand-off, and transfer reports. Nurses routinely report information to other healthcare team members and also urgently contact healthcare providers to report changes in patient status.
Standardized methods of communication have been developed to ensure that information is...
Standardized methods of communication have been developed to ensure that information is...
4.4K
Types of Reports III: Telephone and Verbal Reports
720
Telephone and Verbal Reports in healthcare settings are two communication methods for conveying therapeutic instructions from healthcare providers to nurses or other healthcare staff.
Here's an overview of each type:
Telephone Orders
Here's an overview of each type:
Telephone Orders
720
Methods of Documentation II: POMR
914
The Problem-Oriented Medical Record (POMR) revolutionized medical record-keeping by introducing a systematic approach focusing on the patient's problems rather than merely listing symptoms. Dr. Lawrence Weed's introduction of this method in the 1960s marked a significant advancement in medical documentation. The POMR framework consists of four key components: the database, problem list, plan of care, and progress notes.
914
Guidelines for Nursing Documentation I
1.0K
Quality documentation and reporting share essential characteristics that ensure they are practical and valuable resources for those who use them. These characteristics are:
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
1.0K
Methods of Documentation VI: Case Management Model
559
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
559
