Large language models show human-like content biases in transmission chain experiments

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Cultural Evolution

Background:

• Human cultural transmission exhibits biases, favoring certain content types.
• Large language models (LLMs) are increasingly prevalent and may inherit human biases.

Purpose of the Study:

• To investigate if LLMs, specifically ChatGPT-3, exhibit biases analogous to human cultural transmission.
• To identify the types of content biases present in LLM outputs.

Main Methods:

• Utilized a transmission chain experimental methodology, mirroring human studies.
• Employed preregistered experiments with material from prior human participant studies.
• Analyzed ChatGPT-3 output for content biases.

Main Results:

• ChatGPT-3 demonstrated biases for gender-stereotype-consistent content.
• LLM output showed biases for social, negative, and threat-related information.
• Biases were also observed for biologically counterintuitive content over other types.

Conclusions:

• LLM biases suggest widespread, cognitively appealing content in training data.
• These biases may amplify existing human tendencies for non-informative content.
• Further research is needed on the downstream effects of LLM biases.