对解决临床药房问题的生成人工智能系统进行比较分析:混合方法研究研究
Lulu Li1, Pengqiang Du1, Xiaojing Huang1
1Department of Pharmacy, Fuwai Central China Cardiovascular Hospital, 1 Fuwai Road, Zhengdong New District, Zhengzhou, China, 86 18538298379.
JMIR medical informatics
|July 24, 2025
概括
生成型人工智能在临床药房中表现有前途,但存在诸如高风险错误和推理差距等关键局限性. DeepSeek-R1表现最好,但人类监督对于安全的AI部署至关重要.
科学领域:
- 临床药房 临床药房
- 人工智能的人工智能
- 医疗信息学 医疗信息学
背景情况:
- 生成型人工智能 (AI) 系统在临床药房实践中越来越多地使用.
- 在各种药房环境中对人工智能的有效性,局限性和风险的系统评估是有限的.
研究的目的:
- 量化评估和比较8个主流生成AI系统的性能.
- 评估AI在4个核心临床药房场景中的表现:药物咨询,教育,处方审查和病例分析.
- 用一个多维框架进行全面的AI绩效评估.
主要方法:
- 从现实世界药房实践来源中选择了48个经过临床验证的问题.
- 通过使用标准化提示,测试了8个生成性AI系统 (ERNIE Bot,Doubao,Kimi,Qwen,GPT-4o,Gemini-1.5-Pro,Claude-3.5-Sonnet,DeepSeek-R1) 进行了测试.
- 采用双盲评分设计,有6名经验丰富的临床药剂师在6个维度 (准确性,严格性,适用性,连贯性,简洁性,普遍性) 中评估了反应.
主要成果:
- DeepSeek-R1显示了最高的整体性能,特别是在复杂的任务中 (P<.05).
- 在模型中观察到的显著局限性包括高风险错误 (例如,75%的禁忌被遗漏) 和局部化的缺乏 (例如,90%的抗药性菌株建议使用不适当的抗生素).
- 发现了复杂的推理缺陷,只有一个模型检测出性别诊断矛盾,没有一个模型识别出特定的处方限制.
结论:
- 生成型人工智能作为药剂师辅助工具具有前景,但由于存在重大错误和推理差距,尚未支持自主临床决策.
- 目前,DeepSeek-R1在性能方面处于领先地位,但所有系统都需要对知识更新,复杂推理和输出可解释性进行优化.
- 未来的部署需要优先考虑人类监督,道德保障和安全整合的持续评估框架.
相关概念视频
01:12Analysis of Population Pharmacokinetic Data
387
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
387
01:16Non-equilibrium in the Cell
4.9K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.9K
01:26Pharmacokinetic Models: Comparison and Selection Criterion
150
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
150
01:27Nursing Clinical Information System
867
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
867
01:14Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
228
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
228
01:26Drug Administration and Therapy Phases: Overview
753
Drugs, the chemical agents used in diagnosing, treating, or preventing diseases, undergo a four-phase process of development: pharmaceutic, pharmacokinetics, pharmacodynamics, and therapeutic.
The pharmaceutical phase focuses on leveraging the physicochemical properties of the drug to design and manufacture an effective product. Variants include orally administered tablets or capsules, topical creams or ointments, and parenteral-delivery solutions or emulsions.
The pharmacokinetic phase...
The pharmaceutical phase focuses on leveraging the physicochemical properties of the drug to design and manufacture an effective product. Variants include orally administered tablets or capsules, topical creams or ointments, and parenteral-delivery solutions or emulsions.
The pharmacokinetic phase...
753