轻量级的开源大语言模型与cTAKES用于从排放摘要中提取信息:烟草吸烟状态测试案例
David M Dávila-García1, Matthew J Schuelke1, Adam B Wilcox1
1Institute for Informatics, Data Science & Biostatistics, Washington University School of Medicine in Saint Louis, St. Louis, MO 63110, United States.
轻量级大语言模型 (LLM) 显示出临床信息提取的前景. gpt-oss-20B模型的性能与cTAKES相比,为分析排放总结提供了一个高效的开源替代方案.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 医疗保健中的人工智能
背景情况:
- 从电子健康记录中提取临床信息对于研究和患者护理至关重要.
- 像cTAKES这样的传统自然语言处理 (NLP) 系统需要针对特定任务进行大量的开发和微调.
- 开源大型语言模型 (LLM) 为有效的临床数据分析提供了潜在的替代方案.
研究的目的:
- 为了比较轻量级,开源的LLMs与已建立的cTAKES系统的性能,以从住院出院摘要中提取烟草吸烟状态.
- 在现实世界的临床信息提取场景中评估各种LLM的准确性和效率.
主要方法:
- 两位读者对250份成人出院摘要进行了注释,以说明烟草吸烟状况.
- 六个LLM (包括Llama-3,gpt-oss-20B,MedGemma-27B) 和cTAKES被用来提取吸烟状态.
- 使用加权的F1分数,宏观F1分数和每个类的F1分数来评估表现,与共识注释相比进行非劣势测试.
主要成果:
- 在注释方面,读者之间达成了很好的共识 (κ = 0.91).
- 与cTAKES相比,gpt-oss-20B模型表现不劣,F1分数分别为0.99和0.97 (P < .021).
- LLM大小在2.3-47.3 GB之间,推断时间在每笔记2.5-14.5 秒之间.
结论:
- gpt-oss-20B模型为临床信息提取提供了一个高度准确和高效的开源替代方案.
- 轻量级的LLM可以广泛应用于各种临床信息提取任务,而无需对特定任务进行微调.
更多相关视频
09:07Author Spotlight: Development and Characterization of an In Vitro Model to Study Chronic Cigarette Smoke Exposure and Its Impact on Airway Epithelial Cells in COPD Research
Published on: July 12, 2024
09:50Impact Assessment of Repeated Exposure of Organotypic 3D Bronchial and Nasal Tissue Culture Models to Whole Cigarette Smoke
Published on: February 12, 2015
相关概念视频
Statistical Methods for Analyzing Epidemiological Data
Chronic Obstructive Pulmonary Disease
Smoking is a primary risk factor for COPD, with over 80% of patients having a history of it. Patients typically experience progressive dyspnea or labored breathing, frequent coughing, and recurrent pulmonary infections. Many eventually succumb to respiratory failure, characterized by...
Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One...
