评估使用大型语言模型在急诊室提供临床建议
Christopher Y K Williams1, Brenda Y Miao2, Aaron E Kornblith2,3
1Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, USA. cykw2@doctors.org.uk.
Nature communications
|October 8, 2024
概括
像GPT-4这样的大型语言模型 (LLM) 在医疗保健中显示出潜力,但在临床笔记方面却存在困难. 目前的LLM对紧急部门的建议过于谨慎和不如医生准确.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
- 在医疗保健中的自然语言处理.
背景情况:
- 大型语言模型 (LLM) 在医疗保健应用中提供了变革性的潜力.
- 有限的研究存在于LLM性能与现实世界的临床笔记.
研究的目的:
- 评估LLM (GPT-3.5-turbo和GPT-4-turbo) 在生成紧急部门 (ED) 临床建议的准确性.
- 将LLM绩效与住院医生进行入院状态,放射学调查请求和抗生素处方的比较.
主要方法:
- 一项强大的研究分析了1万个随机选择的急救诊所.
- 使用四种提示策略测试了零射击,GPT-3.5轮机和GPT-4轮机模型.
- 通过将LLM建议与医生决定进行比较来评估绩效.
主要成果:
- 无论是GPT-4-turbo还是GPT-3.5-turbo,都表现出比住院医生更低的准确性 (分别低于8%和24%).
- LLM 倾向于过度谨慎,优先考虑高灵敏度而不是特异性.
- 在评估的三个临床推任务中,准确性各不相同.
结论:
- 虽然临床环境中的LLM评估显示出希望,但目前的表现不足以直接部署作为决策支持工具.
- 在临床实施之前,需要在LLM准确性和决策方面取得重大改进.
- 需要进一步的研究,以优化LLMs复杂的临床任务,并确保可靠的患者护理.
更多相关视频
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.0K
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.1K
相关概念视频
01:26Types of Reports III: Telephone and Verbal Reports
724
Telephone and Verbal Reports in healthcare settings are two communication methods for conveying therapeutic instructions from healthcare providers to nurses or other healthcare staff.
Here's an overview of each type:
Telephone Orders
Here's an overview of each type:
Telephone Orders
724
01:22Cardiopulmonary Resuscitation II: ACLS Airway Management
2
Airway management is a key skill in emergency and critical care settings, as maintaining a clear airway is essential for adequate oxygenation and ventilation.Head Tilt-Chin Lift TechniqueThe head tilt-chin lift maneuver is an essential technique primarily used in patients without suspected cervical spine injuries. To perform this maneuver, one hand is placed on the patient’s forehead, and gentle pressure is applied backward to tilt the head. The fingertips of the other hand are positioned...
2