Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Language and Cognition01:27

Language and Cognition

338
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
338
Stereotype Content Model02:16

Stereotype Content Model

14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
Mouse Models of Cancer Study02:43

Mouse Models of Cancer Study

5.5K
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
5.5K
Language Development01:22

Language Development

329
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
329
  1. Home
  3. Research Domains
  5. Language, Communication And Culture
  7. Language Studies
  9. South-east Asian Languages (excl. Indonesian)
  11. Take Caution In Using Llms As Human Surrogates

Take caution in using LLMs as human surrogates

Yuan Gao1, Dokyun Lee1,2, Gordon Burtch1

  • 1Department of Information Systems, Questrom School of Business, Boston University, Boston, MA 02215.

Proceedings of the National Academy of Sciences of the United States of America
|June 13, 2025

Related Experiment Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

523
Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

423
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.4K

View abstract on PubMed

Summary
This summary is machine-generated.

Large language models (LLMs) show promise but struggle to replicate human behavior in social science research. Their probabilistic nature, unlike human cognition, limits their use as reliable surrogates for human participants.

Area of Science:

  • Artificial Intelligence
  • Social Sciences
  • Computational Social Science

Background:

  • Large language models (LLMs) demonstrate human-like text generation capabilities.
  • LLMs are increasingly proposed as surrogates for human participants in social science research.
  • Fundamental differences exist between LLM processing and human cognition, including embodied experience and survival drives.

Purpose of the Study:

  • To evaluate the capability of LLMs to simulate human behavior in social science research.
  • To assess the reasoning depth of LLMs in economic decision-making tasks.
  • To identify limitations and potential biases when using LLMs as human surrogates.

Main Methods:

  • Utilized the 11-20 money request game to test LLM reasoning.
  • Compared LLM responses against established human behavior distributions.
Keywords:
LLMs as a simulationLLMs as human surrogatesLLMs in social science researchSimulations of human behavior

Related Experiment Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

523
Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

423
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.4K
  • Analyzed various advanced LLM approaches and their performance across different models.

    • Main Results:

    • Most advanced LLM approaches failed to accurately replicate human behavior distributions.
    • Failure modes were diverse and unpredictable, influenced by factors like input language and model safeguarding.
    • Significant discrepancies were observed between LLM outputs and human decision-making patterns.

    Conclusions:

    • LLMs, despite human-like responses, do not reliably simulate human behavior due to fundamental cognitive differences.
    • Caution is advised when employing LLMs as surrogates or for simulating human behavior in research.
    • Further research is needed to understand and mitigate LLM limitations in social science applications.