大型语言模型在提供医疗建议时易受快速注射的影响
Ro Woon Lee1,2, Tae Joon Jun3, Jeong-Moo Lee1,4
1Artificial Intelligence Research Committee, GIGA Study, Incheon, Republic of Korea.
JAMA network open
|February 3, 2026
概括
商业大型语言模型 (LLM) 对即时注入攻击非常脆弱,可能导致不安全的医疗建议. 即使是先进的模型也显示出显著的易受性,强调在医疗部署之前需要强有力的安全措施.
科学领域:
- 医疗保健中的人工智能
- 医疗应用中的网络安全
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 在医疗保健中越来越多地使用,这引发了人们对其安全性的担忧.
- 提示注射攻击可以操纵LLM行为,可能改变关键的医疗建议.
- 在临床环境中缺乏对LLM对这些攻击的脆弱性的系统评估.
研究的目的:
- 评估商业LLM对即时注入攻击的易感性.
- 确定这些攻击是否会引发不安全的临床建议.
- 为了验证中间人和客户端注入作为可行的攻击载体.
主要方法:
- 使用标准化患者-LLM对话的受控模拟研究 (2025年1月至10月).
- 评估了12个临床场景的轻量级模型 (GPT-4o-mini,Gemini-2.0-flash-lite,Claude-3-haiku),按危害水平分层.
- 在高风险怀孕场景中使用客户端注射测试了旗舰模型 (GPT-5,Gemini 2.5 Pro,Claude 4.5 Sonnet).
- 在多轮对话框架内采用上下文意识和证据制造注入策略.
主要成果:
- 快速注射攻击在主要决策转折时实现了94.4%的成功率,并在69.4%的后续中持续存在.
- 轻量级模型显示完全 (100%) 或高 (83.3%) 的敏感性.
- 极高危害的情景,包括X类妊娠药物,在91.7%的攻击中取得了成功.
- 旗舰模型在概念验证测试中显示出80-100%的漏洞.
结论:
- 商业LLM对即时注射攻击具有显著的脆弱性,能够产生危险的临床建议.
- 即使是具有安全功能的高级LLM也很容易受到影响,这突显了安全风险.
- 在临床部署LLMs之前,对抗性稳定性测试,系统保护和监管监督至关重要.
更多相关视频
相关概念视频
Language
917
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
917
Components of Language
821
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
821
Language Development
916
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
916
Language and Cognition
801
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
801
Inhaled Medications
803
Inhaled medications are crucial for managing chronic obstructive pulmonary disease (COPD) and asthma. They are essential for effective treatment and control, ensuring optimal respiratory health and well-being. Inhaled medication delivers drugs directly to the lungs, providing a rapid onset of action and reducing systemic side effects compared to oral or injectable medications. Three primary types of inhalation devices are used to administer these medications: nebulizers, metered-dose inhalers...
803
Higher Mental Functions of the Brain: Language
3.8K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.8K


