评估不同版本的ChatGPT如何与老年脚本一致性测试专家意见保持一致
Matthieu Lilamand1, Théodore Decaix2, Pierre-Antoine Gourraud3
1AP-HP. Nord, INSERM UMR-S 1144, Service de Gériatrie Hôpital Lariboisière-Fernand Widal , Université Paris Cité, 200 Rue du Fbg Saint Denis, 75010, Paris, France. matthieu.lilamand@aphp.fr.
European geriatric medicine
|October 20, 2025
概括
像ChatGPT-4和5这样的高级大型语言模型 (LLM) 与脚本一致性测试 (SCT) 专家的临床推理相似. 这些LLM可以通过模拟病例和支持学习来增强医学教育,尽管需要进一步评估.
科学领域:
- 医疗教育中的人工智能
- 临床推理评估评估
- 大型语言模型 (LLM)
背景情况:
- 大型语言模型 (LLM) 在医学教育中越来越多地用于临床推理培训.
- 脚本一致性测试 (SCT) 是评估不确定性下的临床推理的有效工具.
- 与人类专家相比,评估LLM的表现对于理解其教育实用性至关重要.
研究的目的:
- 在脚本一致性测试 (SCT) 上评估四个ChatGPT版本 (3.5,4,4o和5) 的性能.
- 将ChatGPT版本的临床推理能力与老年医学专家进行比较.
- 确定医学教育中LLMs在临床推理评估方面的潜力.
主要方法:
- 对聊天GPT版本和43名老年医学专家进行了20个覆盖老年病学主题的验证SCT问题.
- 答案使用五分利克尔特等级进行评分,综合专家答案作为参考.
- 通过测量LLM响应与多数专家共识的对齐来分析业绩.
主要成果:
- 聊天GPT-3.5得分为8/20,与专家共识一致,对15%的回复.
- 聊天GPT-4和-5获得更高的分数 (分别为15.6/20和15.2/20),与专家一致,占回答的45%.
- 与GPT-3.5.5.5相比,ChatGPT 4o获得了13.6/20的分数,而GPT-4和5则表现出优越的推理和一致性.
结论:
- 在基于SCT的临床推理中,ChatGPT-4和 -5的表现与老年专家的表现相当.
- 在医学教育方面,LLM具有潜力,有助于案例模拟,自主学习和形成性评估.
- 需要进一步的研究来评估跨不同临床领域的LLM绩效,并解决版本变化.
相关概念视频
Drug Dosing: Geriatric Patients
222
Elderly individuals encompass a diverse population with varying degrees of age-related physiological changes. Defining the elderly presents challenges, as the geriatric population is often arbitrarily categorized as individuals older than 65. However, many individuals in this group lead active and healthy lives, with an increasing number surpassing 85 years and falling into the older elderly category. Physiological changes associated with aging impact performance capacity and homeostatic...
222
Pharmacokinetics in Geriatric Patients: Effect of Age on Drug Excretion
233
In geriatric patients, renal physiology undergoes significant changes, including diminished renal blood flow and a lower glomerular filtration rate (GFR), leading to alterations in medication clearance. Drugs such as aminoglycoside antibiotics, lithium, and digoxin, which rely on glomerular filtration for removal from the body, particularly impact pharmacokinetics. These drugs tend to have slower clearance rates in older adults, necessitating careful dosage considerations.Evaluation of renal...
233
Pharmacodynamics in Geriatric Patients: Effects of Age
185
Age-related pharmacokinetic changes are extensively documented, but understanding age-related pharmacodynamic alterations is relatively limited. This knowledge gap can be partly attributed to the complexity of developing appropriate measures of drug responses compared to bioanalytical methods for determining drug concentrations.Most information regarding age-related differences in human pharmacodynamics originates from cross-sectional studies. However, these studies assume that observed mean...
185


