基于情景的心理学和管理实验的大规模复制,使用大型语言模型
Ziyan Cui1, Ning Li2, Huaikang Zhou1
1Tsinghua University, Beijing, China.
Nature computational science
|July 9, 2025
概括
大型语言模型 (LLM) 在复制心理学研究方面表现有前途,但高估了效果大小,并与敏感主题作斗争. 人类验证对于复杂的社会科学研究至关重要.
科学领域:
- 心理学 心理学 心理学
- 人工智能的人工智能
- 计算社会科学 计算社会科学
背景情况:
- 可复制性是科学进步的基石.
- 大型语言模型 (LLM) 为计算研究提供了新的可能性.
- 关于LLM复制心理发现的能力在很大程度上是未被探索的.
研究的目的:
- 用LLMs评估156个心理实验的可复制性.
- 为了比较LLM主要效应与相互作用效应的表现.
- 调查LLM在社会敏感主题和无效结果上的表现.
主要方法:
- 使用三个最先进的LLM,复制了156个心理实验.
- 对主要和相互作用效应的复制率的分析.
- 在LLM和人文研究之间的效果大小的比较.
- 评估有关种族,性别和伦理学研究的LLM绩效.
主要成果:
- 在主要效应方面,LLM实现了高复制率 (73-81%),在相互作用效应方面取得了中等成功 (46-63%).
- LLM 总是产生比人类研究更大的效果大小.
- 对敏感主题 (种族,性别,伦理) 的研究,复制率明显较低.
- 在高率 (68-83%) 的零发现中,LLM产生了显著的结果,表明潜在的高估值.
结论:
- 在心理学中,LLM显示出试点测试和假设验证的潜力.
- 法律学士学位可以丰富,但不能取代传统的人参与研究.
- 微妙的解释和人类验证对于敏感或复杂的社会研究中LLM至关重要.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
587
05:15The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition
Published on: February 19, 2018
10.9K
相关概念视频
Modeling in Therapy
152
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
152
Language and Cognition
453
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
453
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
103
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
103
