源框架触发了大型语言模型中的系统偏差
Federico Germani1, Giovanni Spitale1,2
1Institute of Biomedical Ethics and History of Medicine, University of Zurich, Zurich, Switzerland.
Science advances
|November 7, 2025
概括
大型语言模型 (LLM) 在文本评估中显示出很高的一致性,但当陈述被定义为来自特定国籍时,这种一致性会受到影响,这揭示了人工智能判断中的系统偏见.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 计算社会科学 计算社会科学
背景情况:
- 大型语言模型 (LLM) 越来越多地用于文本评估.
- 对LLM判断的一致性,偏见和稳定性存在担忧.
- 框架效应对LLM评估的影响需要彻底调查.
研究的目的:
- 评估文本评价中最先进的LLM之间的模式间和模式内协议.
- 调查来源归因 (LLM与人类,国籍) 对LLM判断的影响.
- 为了识别来自框架效应的LLM评估中的潜在偏差.
主要方法:
- 通过四个高级法学士课程,评估了24个不同主题的4800个叙事陈述.
- 总共进行了192,000次个人评估.
- 操纵陈述来源归因于特定国籍的人类作者和其他LLMs.
主要成果:
- 在一般主题评估中,在LLM中观察到高的inter-和intramodel协议.
- 源头归因严重破坏了LLM协议,表明易受框架效应的影响.
- 将陈述归因于中国个人系统地降低了协议得分,特别是在DeepSeek Reasoner模型中.
结论:
- 在评估受源框架影响的文本时,LLM表现出系统的偏见.
- 这些框架效应可能会损害LLM介导的信息系统的中立性和公平性.
- 进一步的研究对于减轻偏见和确保可靠的基于LLM的文本评估至关重要.
更多相关视频
相关概念视频
Framing Effects
7.8K
Information is everywhere and its presentation—such as how and when items are presented—can impact our perceptions and decisions surrounding the info. This broad concept umbrellas framing effects—influences that occur due to the way information is framed in its appearance, whether it’s purely the order or the specific wording of a message. Let’s take a look at numerous ways in which two versions of something can objectively say the same thing, yet we respond in...
7.8K
Language and Cognition
700
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
700
Bias
7.2K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
7.2K
Motivational Bias
304
Cognitive bias results from limitations in thinking and information processing, leading to systematic errors in judgment. Conversely, motivational bias stems from personal desires or emotions, causing distortions in perception to align with self-interest. Motivational bias influences how individuals perceive and attribute causes to events, often shaped by personal needs, goals, and self-esteem preservation. This bias can distort judgment, leading to inaccurate assessments of success, failure,...
304
Fundamental Attribution Error
13.7K
According to some social psychologists, people tend to overemphasize internal factors as explanations—or attributions—for the behavior of other people. They tend to assume that the behavior of another person is a trait of that person, and to underestimate the power of the situation on the behavior of others. They tend to fail to recognize when the behavior of another is due to situational variables, and thus to the person’s state. This erroneous assumption is...
13.7K
Correspondence Bias
184
Correspondence bias, also referred to as the fundamental attribution error, describes the tendency to attribute another person’s behavior to internal characteristics rather than situational influences. This cognitive bias leads individuals to overlook external factors that may be influencing actions, thereby fostering potentially inaccurate assessments of others’ intentions and dispositions.Empirical Evidence for Correspondence BiasResearch has consistently demonstrated the...
184


