探索大型语言模型的潜力,以评估药物对ESC心力衰竭指南的遵守
Noman Dormosh1,2,3,4, Machteld Boonstra1,3, Ameen Abu-Hanna1,2,4
1Department of Medical Informatics, Amsterdam UMC Location University of Amsterdam, Amsterdam, The Netherlands.
JAMIA open
|November 18, 2025
概括
微调大型语言模型 (LLM) 显著提高了它们评估心力衰竭药物坚持性的能力. 针对特定任务的微调对于最佳性能至关重要,因为一般的LLM在没有它的情况下表现不佳.
科学领域:
- 人工智能在医学中的应用
- 临床信息学 临床信息学
- 药物治疗研究 药物治疗研究
背景情况:
- 评估临床医生遵守心力衰竭药物治疗指南对于患者的治疗结果至关重要.
- 目前用于评估坚持的方法可能是劳动密集型和耗时的.
- 大型语言模型 (LLM) 提供了自动化评估过程的潜力.
研究的目的:
- 评估各种开源大型语言模型 (LLM) 在自动化评估临床医生遵守欧洲心脏病学会 (ESC) 心力衰竭药物治疗指南方面的有效性.
- 为了比较一般领域的LLM与医学特定的LLM在这个任务中的表现.
- 确定不同提示策略和微调对LLM绩效的影响.
主要方法:
- 利用了1,141名住院心力衰竭患者的电子健康记录 (EHR).
- 根据指导方针建议和目标剂量,开发了一个分类系统来标记药物坚持.
- 评估了六个开源的LLM (三个通用领域,三个医疗特定领域) 使用零射击,少数射击和思维链提示.
- 微调精选的LLM使用单体偏好优化没有参考模型 (ORPO) 方法与合成的偏好数据.
- 进行学习曲线分析以确定最佳的培训数据大小.
- 使用宏 F1 评分来评估业绩.
主要成果:
- 在微调之前,所有LLM的性能都很低 (F1<0.40),这表明几乎是随机猜测.
- 在对特定任务进行了微调后,四名LLM获得了≥0.90.的宏观F1分数.
- 两个LLM,Llama3-8B-Instruct和OpenBioLLM-8B,在微调后分别获得了0.794和0.787的F1分数.
- GLM-4-9B-Chat和Mistral-7B-Instruct-v0.2需要分别40%和50%的训练数据,以达到峰值性能.
- 医学专业的LLM没有显示出与一般领域的LLM相比的显著优势.
结论:
- 针对特定任务的微调对于在LLM中实现高性能至关重要,用于评估心力衰竭药物治疗的坚持.
- 选择合适的LLM及其随后的微调对于成功实施至关重要.
- 在没有微调的情况下,无论其领域特异性如何,LLM在适应专门的临床任务方面都存在重大限制.
相关概念视频
01:28Drug Therapy
226
The advent of drug therapy has profoundly shaped modern mental health care, providing targeted treatments for a range of psychological disorders. Psychotherapeutic drugs, classified into antianxiety, antidepressant, and antipsychotic medications, address symptoms across anxiety disorders, mood disorders, and schizophrenia. While these medications have transformed patient outcomes, they require careful management due to their potential side effects and limitations.
Antianxiety Medications
Antianxiety Medications
226
01:30Heart Failure V: Medical Management
195
Medical Management of Acute Decompensated Heart Failure (ADHF)The primary goals of therapy for patients hospitalized with acute decompensated heart failure (ADHF) include:Relieving symptomsOptimizing volume statusSupporting oxygenation and ventilationMaintaining cardiac output (CO) and end-organ perfusionIdentifying and addressing the cause of ADHFPreventing complicationsProviding patient education on factors precipitating HF exacerbationPlanning for dischargeOngoing monitoring and assessment...
195
01:26Heart Failure Drugs: Inhibitors of Renin-Angiotensin System
869
The activation of the sympathetic nervous system and the renin-angiotensin-aldosterone system (RAAS) contributes to cardiac remodeling, and inhibiting the RAAS is a pharmacological target in heart failure management. As a result, neurohumoral modulation is a crucial treatment principle for managing heart failure. This approach involves using medications like ACE inhibitors (ACEIs), angiotensin receptor blockers (ARBs), β-blockers, mineralocorticoid receptor antagonists (MRAs), and neutral...
869
01:22Heart Failure VI: Adjunct Therapies
228
Additional therapies for treating patients with heart failure (HF) may include procedural interventions, supplemental oxygen, the management of sleep disorders, and nutritional therapy.Procedural InterventionsImplantable Cardioverter-Defibrillator: For patients at risk of life-threatening arrhythmias due to severe left ventricular dysfunction, an Implantable Cardioverter-Defibrillator (ICD) can detect and terminate these arrhythmias, preventing sudden cardiac death and improving survival rates.
228
01:22Heart Failure Drugs: β-Blockers
724
β-adrenergic antagonists, commonly known as β-blockers, block the effects of sympathetic neurotransmitters such as noradrenaline (NA) and adrenaline (ADR). They have several beneficial effects in heart failure treatment. They reduce heart rate, the force of contraction, and cardiac muscle relaxation. They also slow the atrial-ventricular conduction rate and raise the threshold for arrhythmias. The concentration of β-blockers determines their effects on bronchodilation,...
724
01:15Drug Dosing: Geriatric Patients
199
Elderly individuals encompass a diverse population with varying degrees of age-related physiological changes. Defining the elderly presents challenges, as the geriatric population is often arbitrarily categorized as individuals older than 65. However, many individuals in this group lead active and healthy lives, with an increasing number surpassing 85 years and falling into the older elderly category. Physiological changes associated with aging impact performance capacity and homeostatic...
199