在心血管抗血栓治疗中评估大型语言模型:性能,准确性和临床实践的影响
Pavel Antiperovitch1, Iris Liu2, Ahmed T Mokhtar1
1Department of Medicine, Division of Cardiology, London Health Sciences Centre, Western University, London, Ontario, Canada.
The Canadian journal of cardiology
|April 16, 2025
概括
大型语言模型 (LLM) 在心血管护理场景中表现不同. 克劳德3Opus的表现优于临床医生,但所有模型都可接受的生活方式建议,需要谨慎使用和验证.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
- 心血管医学 心血管医学
背景情况:
- 大型语言模型 (LLM) 在医疗保健中越来越多地使用,但人们对其在患者管理中的准确性存在担忧.
- 之前的研究已经强调了在临床环境中个别LLM的潜在不准确性.
研究的目的:
- 评估公开可用的大型语言模型 (LLM) 在心血管抗血栓治疗场景中的性能.
- 在模拟的临床决策中,将LLM绩效与人类临床医生进行比较.
主要方法:
- 根据验证的心血管抗血栓治疗方案,评估了七种公开可用的LLM.
- 三位独立的临床医生评估了LLM的准确性和推理,将结果与志愿临床医生的调查进行了比较.
- 使用统计分析来确保观察者之间的可靠性,并评估性能差异.
主要成果:
- 克劳德3Opus的准确率达到了85%,明显超过了其他LLM和所有临床医生群体.
- 心脏病专家和高级住院医生表现出最高的临床准确度 (43-47%),与GPT-4o (55%) 和克劳德3.5索内特 (44%) 相比.
- 一般医生和医学学生的表现不太准确,一些自由级模型提供了不适当的建议.
结论:
- 在心血管情景中LLM的性能差异很大,一些模型超过了临床医生的准确性.
- 所有经过测试的LLM均表现出符合生活方式和饮食建议的可接受性能.
- 临床医生和患者应谨慎使用LLM,选择合适的模型,并核实安全实践信息.
相关概念视频
Anticoagulant Drugs: Vitamin K Antagonists and Direct Oral Anticoagulants
1.0K
Oral anticoagulants are vital tools in preventing and treating blood clotting disorders. This diverse class of medications can be categorized as vitamin K antagonists, exemplified by warfarin, and direct thrombin inhibitors (DTIs), such as dabigatran, as well as factor Xa inhibitors, including rivaroxaban.
Warfarin, a prominent vitamin K antagonist family member, exerts its effect by inhibiting the enzyme VKORC1 (vitamin K epoxide reductase complex 1). By hindering this enzyme, warfarin...
Warfarin, a prominent vitamin K antagonist family member, exerts its effect by inhibiting the enzyme VKORC1 (vitamin K epoxide reductase complex 1). By hindering this enzyme, warfarin...
1.0K
Cardiovascular Drugs: Classification based on Therapeutic Indications
1.9K
Cardiovascular diseases, encompassing a range of conditions, can significantly affect the heart's operations and the overall circulatory system. These conditions impair the heart's ability to pump blood, leading to a deficit in oxygen supply to crucial organs. Anomalies in the heart's electrical system, known as arrhythmias, can cause heartbeats to accelerate or slow down. Usually, heart rates increase during physical activity and decrease while resting or sleeping. However,...
1.9K
Anticoagulant Drugs: Low-Molecular-Weight Heparins
546
Hemostasis is a crucial process that prevents excessive blood loss from damaged blood vessels. It involves various mechanisms such as vasoconstriction, platelet adhesion and activation, and fibrin formation. The importance of each mechanism depends on the type of vessel injury. In contrast, thrombosis is the abnormal formation of a blood clot within the blood vessels, leading to potential complications if the clot obstructs blood flow. Thrombosis can be caused by increased coagulability of the...
546
Antiplatelet Drugs: Prostaglandin Synthesis, P2Y12 and Glycoprotein IIb/IIIa Inhibitors
392
Antiplatelet drugs emerge as frontline defenders against the insidious threat of thromboembolic diseases, where abnormal clots obstruct vital blood vessels. These drugs stand as bulwarks, inhibiting platelet aggregation and clot formation, thereby mitigating the risk of life-threatening conditions like myocardial infarction, coronary artery disease, and thrombotic strokes.
Prostaglandin synthesis inhibitors, exemplified by the widely known aspirin, wield their power by irreversibly acetylating...
Prostaglandin synthesis inhibitors, exemplified by the widely known aspirin, wield their power by irreversibly acetylating...
392
Heart Failure Drugs: Inhibitors of Renin-Angiotensin System
316
The activation of the sympathetic nervous system and the renin-angiotensin-aldosterone system (RAAS) contributes to cardiac remodeling, and inhibiting the RAAS is a pharmacological target in heart failure management. As a result, neurohumoral modulation is a crucial treatment principle for managing heart failure. This approach involves using medications like ACE inhibitors (ACEIs), angiotensin receptor blockers (ARBs), β-blockers, mineralocorticoid receptor antagonists (MRAs), and neutral...
316


