在两个机构的电子健康记录中的临床推理文档的基于大型语言模型的评估:开发和验证研究
Verity Schaye1,2, David DiTullio1, Benedict Vincent Guzman3
1Department of Medicine, NYU Grossman School of Medicine, New York, NY, United States.
Journal of medical Internet research
|March 21, 2025
概括
人工智能,包括大型语言模型 (LLM),现在可以评估电子健康记录中的临床推理 (CR) 文档. 这项研究表明,人工智能工具可以改善医生对基本CR技能的反.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床推理评估评估
背景情况:
- 临床推理 (CR) 是医生的关键技能,但电子健康记录 (EHR) 中对其文档的反往往有限.
- 人工智能 (AI) 是一个有希望的解决方案,可以弥合医学教育和实践中的反差距.
研究的目的:
- 开发和评估人工智能驱动的评估,用于电子健康记录中的临床推理 (CR) 文档.
- 在两个机构中比较命名实体识别 (NER),基于逻辑的模型和大型语言模型 (LLM) 的性能.
主要方法:
- 利用了从纽约大学和加州大学的内部医学住院招生笔记的回顾性和前性语料库.
- 开发和比较人工智能模型,包括基于逻辑的模型NER和两种LLM (NYUTron和GatorTron) 用于CR评估.
- 使用F1分数,AUROC和AUPRC对差异诊断和推理领域的解释来评估模型性能.
主要成果:
- 在纽约大学,NYUTron LLM在评估差异诊断 (D0,D2) 和解释推理 (EA2) 中表现强.
- 在UC,基于NER的逻辑模型在差异诊断评估 (D0,D1,D2) 中表现出色,而GatorTron LLM在EA2中表现最好.
- 模型性能因任务和机构而异,有些需要适应性策略,如逐步方法或二进制分类.
结论:
- 这项多机构研究是首次将LLMs应用于评估EHR临床推理文档.
- 开发的AI工具显示了增强医生对临床推理的反的潜力,支持医学教育和实践.
- 学到的经验教训表明,这些人工智能方法在不同的机构环境中具有普遍性.
更多相关视频
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.0K
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
15.7K
相关概念视频
Methods of Documentation V: CBE
863
Charting by Exception, or CBE, is a method of documentation used in healthcare, particularly in nursing, that focuses on documenting only significant or abnormal findings rather than recording every detail. This approach aims to streamline the documentation process, improve efficiency, and ensure that healthcare providers can quickly identify deviations from normalcy in patient assessments.
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
863
Methods of Documentation VII: EMR
821
Electronic Medical Records (EMRs) primarily center around electronically documenting patients' health information within a single healthcare organization or practice. They contain essential clinical data related to a patient's medical history, diagnoses, medications, treatment plans, lab results, and other pertinent information relevant to the specific encounter or episode of care. EMRs are designed to streamline documentation and workflow processes within individual healthcare...
821
Methods of Documentation VI: Case Management Model
556
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
556
Documentation of Nursing Diagnosis
1.2K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.2K
Methods of Documentation III: PIE
1.3K
Problem-intervention-evaluation (PIE) is a systematic approach to documentation used in healthcare settings for clinical decision-making and patient care planning. It is a structured approach to organizing patient data based on problems, interventions, and evaluations. Here's a breakdown of its key features and considerations:
1.3K
Patient-centered Care
1.9K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
1.9K
