Related Concept Videos
Framing Effects
Language and Cognition
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Motivational Bias
Fundamental Attribution Error
Correspondence Bias
You might also read
Related Articles
Articles linked to this work by shared authors, journal, and citation graph.
Analyzing Misinformation and Disinformation: Understanding Swiss COVID-19 Narratives Through Natural Language Processing Analysis.
Financial Burden in Adults With Chronic Illness in Switzerland: A Secondary Analysis of Qualitative Interviews Using Natural Language Processing and Topic Modeling.
Why ethics in social listening and infodemic management matters for public health.
Trump's statements about acetaminophen and the problem of epistemic corrections.
Healthcare resource allocation for rare diseases: an exploratory survey of Swiss citizens' preferences.
Taphonomic analysis at Liang Bua reveals the behavioral and technological capabilities of <i>Homo floresiensis</i>.
Targeting granule initiation and amyloplast structure to create giant starch granules in wheat.
A meta-analysis of carbon losses and gains from tropical moist forest degradation and regeneration.
Ancient DNA reveals elite dynastic rule among Iron Age Eurasian Steppe nomads.
Targeting astrocytic Dp71 attenuates BBB disruption after traumatic brain injury through WTAP-associated m<sup>6</sup>A regulation of MMP2.
Pancreatic α cells are required for nutrient homeostasis by regulating dynamic β cell networks in islets.
Related Experiment Video
Updated: Jan 12, 2026

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Source framing triggers systematic bias in large language models.
Federico Germani1, Giovanni Spitale1,2
1Institute of Biomedical Ethics and History of Medicine, University of Zurich, Zurich, Switzerland.
Large language models (LLMs) show high agreement in text evaluation, but this consistency falters when statements are framed as originating from specific nationalities, revealing systematic bias in AI judgments.
More Related Videos
Area of Science:
- Artificial Intelligence
- Natural Language Processing
- Computational Social Science
Background:
- Large language models (LLMs) are increasingly utilized for text evaluation.
- Concerns exist regarding the consistency, bias, and robustness of LLM judgments.
- The impact of framing effects on LLM evaluations requires thorough investigation.
Purpose of the Study:
- To assess inter- and intramodel agreement among state-of-the-art LLMs in text evaluation.
- To investigate the influence of source attribution (LLM vs. human, nationality) on LLM judgments.
- To identify potential biases in LLM evaluations stemming from framing effects.
Main Methods:
- Evaluated 4800 narrative statements across 24 diverse topics using four advanced LLMs.
- Conducted a total of 192,000 individual assessments.
- Manipulated statement source attribution to human authors of specified nationalities and other LLMs.
Main Results:
- High inter- and intramodel agreement was observed across LLMs for general topic evaluations.
- Source attribution significantly disrupted LLM agreement, demonstrating susceptibility to framing effects.
- Attributing statements to Chinese individuals systematically reduced agreement scores, particularly for the DeepSeek Reasoner model.
Conclusions:
- LLMs exhibit systematic bias when evaluating text influenced by source framing.
- The neutrality and fairness of LLM-mediated information systems are potentially compromised by these framing effects.
- Further research is crucial to mitigate biases and ensure reliable LLM-based text evaluation.

