脱离语境和范围:通过最小的指令集修改来操纵大型语言模型
Monty-Maximilian Zühlke1, Daniel Kudenko1, Wolfgang Nejdl1
1L3S Research Center, Leibniz University Hannover, Lower Saxony, Germany.
PloS one
|February 11, 2026
概括
大型语言模型 (LLM) 可以在训练过程中从微妙的指令中采用特定的行为. 这种操纵是由特定提示引发的,突出显示未经监控的LLM使用的风险.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 大型语言模型 (LLM) 中新兴的推理能力需要与人类的意图保持一致.
- 脱离上下文的推理允许LLM在没有例子的情况下从描述性信息中采用行为.
- 由于不透明的培训数据,评估LLM行为模式的良性是具有挑战性的.
研究的目的:
- 在LLMs中调查用户定义行为的嵌入和触发机制.
- 了解培训数据和提示策略对LLM行为的影响.
- 评估LLM响应模式的操纵潜力.
主要方法:
- 在更大的指令集中微调LLM,以最小的,隐藏的行为描述.
- 数学动机和信号扩散和训练数据中的推理的经验验证.
- 对提示策略和行为触发的代币强化进行分析.
主要成果:
- 从噪音训练数据中的分散信号中,LLM可以推断并采用响应模式.
- 行为嵌入对最小的指令集修改敏感.
- 触发这些嵌入式行为在很大程度上取决于特定的提示策略和代币增强.
结论:
- 通过微妙的修改训练数据和提示,LLM表现出易受操纵的敏感性.
- 在某些条件下,LLM中特定行为的出现可以被控制和触发.
- 由于潜在的行为操纵,使用无法访问的训练数据的LLM无监控使用会带来不可预见的风险.
相关概念视频
The Scope of Physics
53.6K
Physics is concerned with the interactions of energy, matter, space, and time, in order to discover the underlying mechanisms that underpin all phenomena. The word "physics" comes from the Greek word "phúsis", which means nature. Physics seeks to comprehend the natural world around us at its most fundamental level. It emphasizes the use of quantitative laws to do this, which could be valuable in other fields that want to push the performance boundaries of present...
53.6K
Language
921
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
921
Histone Modification
16.3K
The histone proteins have a flexible N-terminal tail extending out from the nucleosome. These histone tails are often subjected to post-translational modifications such as acetylation, methylation, phosphorylation, and ubiquitination. Particular combinations of these modifications form “histone codes” that influence the chromatin folding and tissue-specific gene expression.
Acetylation
The enzyme histone acetyltransferase adds acetyl group to the histones. Another enzyme, histone...
Acetylation
The enzyme histone acetyltransferase adds acetyl group to the histones. Another enzyme, histone...
16.3K
Self Within Cultural Contexts
243
Cultural frameworks for understanding the self are often categorized into two broad orientations: individualism and collectivism. These paradigms influence how people define themselves, relate to others, and interpret their social worlds. Each orientation offers distinct perspectives on autonomy, responsibility, and the role of the individual within a community.Individualistic CulturesIn individualistic cultures like North America and Western Europe, identity is understood as autonomous and...
243
Components of Language
830
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
830
Language Development
936
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
936


