一个数据集和对医院课程总结的基准,采用适应的大型语言模型
Asad Aali1,2, Dave Van Veen3,4, Yamin Ishraq Arefeen2
1Department of Radiology, Stanford University, Stanford, CA 94304, United States.
Journal of the American Medical Informatics Association : JAMIA
|January 9, 2025
概括
大型语言模型 (LLM) 现在可以从临床笔记中合成简要的医院课程 (BHC) 摘要. GPT-4在临床读者研究中表现出卓越的表现,突出了LLM在医疗保健总结方面的潜力.
科学领域:
- 人工智能在医学中的应用
- 临床信息学 临床信息学
- 自然语言处理自然语言处理.
背景情况:
- 简要的医院课程 (BHC) 摘要是关键的临床文件.
- 使用大语言模型 (LLM) 来自临床笔记自动化BCH合成是一个新兴的领域.
- 医疗保健总结现有的LLM能力需要进一步调查.
研究的目的:
- 引入MIMIC-IV-BHC数据集,以适应LLM与BHC合成.
- 为了对一般用途和适合医疗保健的LLMs的总结性能进行基准测试.
- 评估LLM产生的BHC,以提高临床决策.
主要方法:
- 使用临床笔记作为LLMs的输入.
- 应用基于提示和基于微调的适应策略.
- 通过使用定量指标 (BLEU,BERT-Score) 和5名临床医生进行的定性临床读者研究来评估LLM.
主要成果:
- 精心调整的Llama2-13B在定量指标上表现优于其他适应领域的模型.
- 具有上下文学习的GPT-4显示出对增加上下文长度的稳定性.
- 临床医生显著更喜欢GPT-4生成的摘要,而不是精心调整的Llama2-13B和原始摘要 (P<.001).
结论:
- 发布了MIMIC-IV-BHC数据集和LLM绩效基准.
- 专有和开源的LLM都表现出高质量的总结性能.
- 由LLM生成的摘要,特别是来自GPT-4,显示了改善临床决策的潜力.
相关概念视频
00:59Hospitals-II
730
Hospitals provide inpatient and outpatient services. Inpatient services provide care to patients that stay in the hospital for an extended period, ranging from days to months. Examples of inpatient services include intensive care units, hospital wards, or surgeries. Outpatient services provide care to patients who come to a hospital for a diagnostic or treatment but do not stay overnight —for example, diagnostic tests, surgical procedures, or health education.
Nurses that work in...
Nurses that work in...
730
02:07Improving Translational Accuracy
2.5K
2.5K
01:28Hospitals-I
790
Hospitals offer medical and surgical care to the sick and injured, along with accommodation while they recover. At the same time, they also provide outpatient, emergency, psychiatric, and rehabilitation services to meet various community needs. In addition to providing medical care, hospitals also act as hubs for medical research and training. Hospitals use clinical procedures and evidence-based practice standards to deliver patient care. To deliver safe and efficient care, a nurse must stay up...
790
01:12Statistical Software for Data Analysis and Clinical Trials
491
Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
491
01:24Kaplan-Meier Approach
90
The Kaplan-Meier estimator is a non-parametric method used to estimate the survival function from time-to-event data. In medical research, it is frequently employed to measure the proportion of patients surviving for a certain period after treatment. This estimator is fundamental in analyzing time-to-event data, making it indispensable in clinical trials, epidemiological studies, and reliability engineering. By estimating survival probabilities, researchers can evaluate treatment effectiveness,...
90
01:09Truncation in Survival Analysis
156
Truncation in survival analysis refers to the exclusion of individuals or events from the dataset based on specific criteria related to the time of the event. This exclusion can happen in two primary forms: left truncation and right truncation.
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
156