为什么大型语言模型的临床推理失败:可解释的深度学习的见解
medRxiv : the preprint server for health sciences
|February 6, 2026
概括
医学大语言模型 (LLM) 显示不稳定的临床推理,尽管基准得分高. 不同的模型架构独特地编码医疗术语,需要架构特定的安全验证才能可靠地部署AI.
科学领域:
- 医疗人工智能 医疗人工智能
- 临床决策支持系统 临床决策支持系统
- 医学中的计算语言学
背景情况:
- 医学大语言模型 (LLM) 显示出高基准准确度,但表现出无法解释的临床推理变化和错误.
- 稀少的自动编码器提供了一种机械解释性方法,以了解医学中的LLM知识表示和失败模式.
- 现有的基准可能无法捕捉到安全部署人工智能所需的临床推理稳定性的全部范围.
研究的目的:
- 在系统性干扰下评估不同医学LLM架构 (GPT-5,MedGemma-27B-Text-IT,OpenBioLLM-Llama3-70B) 的临床推理稳定性.
- 分析不同的模型架构如何使用稀疏的自编码器和消去实验编码多种医学术语.
- 评估检索干预措施的有效性,以消除医学术语的含义和改善模型性能.
主要方法:
- 在瘤病例中使用355个系统性干扰来评估推理稳定性,将分期和治疗与NCCN和AJCC指南进行比较.
- 在MIMIC-IV临床笔记中的10亿个令牌上训练了稀疏的自编码器,以分析多种医学术语的编码.
- 进行了850次废除实验,并测试了两阶段的检索干预,以获得感觉清晰度.
主要成果:
- 模型显示了显著的推理不稳定性;根据提示格式,OpenBioLLM的分期精度从45.9%到99.1%不等.
- 稀缺的自编码器分析显示了编码中的显著差异:MedGemma显示了77.8%的字体感觉功能重叠,OpenBioLLM 13.6%.
- 检索干预提高了MedGemma的模糊性10.2%,但损害了OpenBioLLM的2.0%,表明了架构特定的干预效应.
结论:
- 医疗人工智能系统表现出临床推理脆弱性,而不是通过基准性能来捕捉,这凸显了对更深层次的解释能力的需求.
- 建筑上不同的模型以不同的方式编码医疗概念,这意味着对一个有效的干预措施可能会伤害另一个.
- 医疗人工智能的安全验证必须是架构特定的,因为基准等价性不能保证功能等价性.
相关概念视频
Pharmacokinetic Models: Overview
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal assumptions,...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal assumptions,...
Mechanistic Models: Overview of Compartment Models
Mechanistic models, a category encompassing both physiological and compartmental modeling, differ from empirical models' approaches to incorporating known factors about the systems being modeled. Empirical models describe data with minimal assumptions, while mechanistic models aim to provide a robust description of available data by specifying assumptions and integrating known factors about the system. Compartmental analysis is a key example of a mechanistic model in pharmacokinetics and...
Mechanistic Models: Compartment Models in Individual and Population Analysis
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least squares (OLS)...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Model Approaches for Pharmacokinetic Data: Physiological Models
Physiological models in pharmacokinetics are instrumental in understanding the distribution and elimination of drugs within the body. These models describe the drug concentration within target organs, influenced by factors such as drug uptake, tissue volume, and blood flow. Drug uptake is governed by the partition coefficient, which signifies the drug concentration ratio in tissue to that in the blood. The blood flow rate to a specific tissue is expressed as Qt, and the rate of change in tissue...
Pharmacokinetic Models: Comparison and Selection Criterion
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.

