当代理的LLM信任有毒的工具时:临床LLM对对手指南的脆弱性
Research square
|February 27, 2026
概括
代理大型语言模型 (LLM) 难以拒绝修改后的医疗指南,经常选择错误的信息. 这一漏洞带来了风险,特别是当人工智能代理人充当主要的健康守护者时.
科学领域:
- 人工智能在医学中的应用
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 代理大型语言模型 (LLM) 越来越多地与外部工具和数据源集成.
- 在从可能受到损害的来源中选择准确信息时,LLM的可靠性仍然是一个开放的问题.
研究的目的:
- 评估21名LLM在辨别真实医疗指南与反向修改版本方面的能力.
- 为了确定失败率和影响因素在医学决策LLM工具的选择.
主要方法:
- 21个LLM在12个领域的500个医生验证的医学图片上进行了测试.
- 模型在真实和假的 (被对方修改的) 准则摘录之间做出选择.
- 总共分析了10,500个代理决策,考虑了呈现顺序.
主要成果:
- 在40.6%的案例中,LLM选择了虚假的,不正确的指南,达到59.4%的准确率.
- 安全关键的修改 (54.2%-61.7%) 的失败率最高,包括改变警告,过敏信息,禁忌和剂量.
- 模型选择在很大程度上受到呈现偏差的影响,有利于第一个呈现的选项.
结论:
- 代理法学士指南选择易受中毒来源的影响,因此在临床部署之前需要采取保障措施.
- 独立的验证和排名机制对于确保医疗保健中的AI代理商可靠性至关重要.
- 依赖人工智能代理的低资源设置面临来自不可靠工具的高风险.
相关概念视频
Avoidance Learning and Learned Helplessness
2.8K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.8K
Improving Translational Accuracy
15.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.3K
Improving Translational Accuracy
3.7K
3.7K
Strategies for Assessing and Addressing Confounding
489
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
489
Pharmaceutical Poisoning: Potential Scenarios
39
Pharmaceutical poisoning can occur through various channels, impacting an estimated 2 million hospitalized patients in the U.S. annually with serious adverse drug responses. These scenarios encompass both therapeutic uses, such as drug toxicity, where even standard dosages can lead to severe central nervous system depression, and non-therapeutic exposures, including accidental ingestion by children, and environmental and occupational exposures.Unintentional poisonings often involve exploratory...
39
Language and Cognition
881
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
881

