提示式架构在大型语言模型中诱导了方法学工件
Melanie Brucks1, Olivier Toubia1
1Columbia Business School, New York, NY, United States of America.
PloS one
|April 28, 2025
概括
提示架构显著偏差大语言模型 (LLM) 响应. 在多个提示符中汇总结果有效地消除了这些方法论文物,揭示了LLM的错误性.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 计算语言学 计算语言学
背景情况:
- 大型语言模型 (LLM) 越来越多地用于各种任务.
- 提示函数结构的方式,称为"提示函数架构",可以影响LLM输出.
- 方法学文物,一种形式的统计偏见,可能来自于快速设计.
研究的目的:
- 调查提示架构如何影响LLM的响应 (GPT-3,GPT-4,Llama 3.1).
- 识别导致偏差的特定提示特征 (顺序,标签,框架,理由).
- 测试缓解这些提示诱导偏见的策略.
主要方法:
- 通过使用标准的零射击相似性评估任务进行了五次大规模的全因数实验.
- 分析了即时订单,标签,框架和理由对LLM响应的影响.
- 经过测试的缓解策略,包括随机化规范和跨提示的聚合.
主要成果:
- 提示架构元素 (顺序,标签,框架,理由) 强烈影响跨模型的LLM响应.
- 法律法规表现出响应顺序偏差和标签偏差,通过框架和理由进行调节.
- 从完整的因数设计中汇总结果有效地消除了响应顺序和标签偏差.
结论:
- 由于与嵌入式语言数据的微妙相互作用,个别提示本质上是错误的.
- 快速设计特征在LLM输出中显著引入统计偏差.
- 在各种提示中聚合是克服LLMs中提示诱导偏见的关键策略.
相关概念视频
Language and Cognition
287
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
287
Concepts and Prototypes
52
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
52
Language Development
280
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
280
Components of Language
194
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
194
Elaborative Rehearsals
58
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
58
Modeling in Therapy
26
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
26


