越狱和守卫对齐的语言模型,只有少数背景示范
IEEE transactions on pattern analysis and machine intelligence
|February 2, 2026
概括
大型语言模型 (LLM) 可以使用上下文学习 (ICL) 来操纵. 新的方法,上下文攻击 (ICA) 和上下文防御 (ICD),通过演示展示如何利用或增强LLM安全对齐.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 机器学习安全 机器学习安全
背景情况:
- 大型语言模型 (LLM) 显示出巨大的潜力,但容易受到恶意利用,特别是越狱攻击.
- 敌对输入可以导致LLM产生有害或不道德的内容,这对安全部署构成重大挑战.
- 语境学习 (ICL) 是LLM的有效和可扩展的技术,为解决安全问题提供了一种新的方法.
研究的目的:
- 调查在语境学习 (ICL) 的潜力,以调整大型语言模型 (LLM) 的安全调整.
- 引入和评估在ICL框架内使用对抗性演示来颠覆和增强LLM安全的方法.
- 探索ICL对LLM安全性和调整的理论和实证影响.
主要方法:
- 拟议的上下文攻击 (ICA):利用有害的演示来损害LLM的安全调整.
- 拟议的上下文防御 (ICD):采用拒绝有害提示的例子来加强LLM的弹性.
- 在多个LLM架构,数据集和攻击基线上进行理论分析和经验验证.
主要成果:
- 证明了最小的上下文示范可以有效地改变LLM安全对齐.
- ICA有效地破坏了LLM的安全性,而ICD显著增强了对有害输出的弹性.
- ICA和ICD都在红色团队评估和制定保障措施方面表现出有效性和可扩展性.
结论:
- 在上下文学习 (ICL) 在LLM安全和调整中发挥着关键作用,这是以前研究不足的因素.
- 拟议的ICA和ICD方法为了解和操纵LLM安全提供了新的工具.
- 这项研究为提高LLM安全性和开发针对对抗性攻击的强有力的防御提供了新的途径.
更多相关视频
06:16Involving Individuals with Developmental Language Disorder and Their Parents/Carers in Research Priority Setting
Published on: June 6, 2020
4.6K
16:02Demonstration of the Sequence Alignment to Predict Across Species Susceptibility Tool for Rapid Assessment of Protein Conservation
Published on: February 10, 2023
3.3K
相关概念视频
Language
913
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
913
Self Within Cultural Contexts
237
Cultural frameworks for understanding the self are often categorized into two broad orientations: individualism and collectivism. These paradigms influence how people define themselves, relate to others, and interpret their social worlds. Each orientation offers distinct perspectives on autonomy, responsibility, and the role of the individual within a community.Individualistic CulturesIn individualistic cultures like North America and Western Europe, identity is understood as autonomous and...
237
Components of Language
821
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
821
Language Development
912
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
912
Language and Cognition
800
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
800
Impact of Social Context on Individuals
332
Social psychology examines how the real or imagined presence of others influences individuals' thoughts, feelings, and behaviors. A key concept in this field is the role of social context in shaping behavior. The same individual may act differently depending on the social setting, due to the varying expectations and norms associated with each environment. This context-dependent behavior illustrates the influence of social roles, which prescribe appropriate conduct in specific situations.Social...
332
