在大型语言模型和人类中测试思想理论
James W A Strachan1, Dalila Albergo2,3, Giulia Borghini2
1Department of Neurology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany. james.wa.strachan@gmail.com.
Nature human behaviour
|May 20, 2024
概括
大型语言模型 (LLM) 在某些任务中显示出类似人类的思维理论能力,GPT-4在错误信念等领域匹配或超过人类表现. 然而,LLM和人类都难以检测错误,这表明人工智能和人类认知的细微差别.
科学领域:
- 认知科学 认知科学
- 人工智能的人工智能
- 计算语言学 计算语言学
背景情况:
- 思想理论 (ToM) 是人类的一个关键认知能力,涉及跟踪他人的心理状态.
- 大型语言模型 (LLM) 越来越复杂,促使与人类认知功能进行比较.
- 关于LLM是否可以在ToM任务中表现出类似人类的行为,存在争论.
研究的目的:
- 在一个全面的ToM测试套件上系统地比较人类和LLM的表现.
- 在各种 ToM 维度中调查 GPT 和 LLaMA2 家庭的 LLM 能力.
- 了解人类和法学士之间的ToM性能差异和相似性的性质.
主要方法:
- 对1,907名人类参与者进行了一组ToM测量.
- 在相同的ToM措施中反复测试了两个LLM家族 (GPT和LLaMA2).
- 包括评估错误信念,间接请求,刺和错误检测的任务.
主要成果:
- 在间接请求,错误信念和误导下,在人类水平或以上执行GPT-4模型.
- 只有在错误传递检测方面,LLaMA2的表现优于人类,后来发现这是一个虚幻的优势.
- GPT-4的局限性源于保守的推断方法,而不是根本缺乏能力.
结论:
- 在某些 ToM 方面,LLM 表现出与精神推理一致的行为,与人类相比.
- 系统的,细微的测试对于人类和人工智能之间的准确比较至关重要.
- 绩效差异凸显了ToM的复杂性以及对LLM认知的进一步研究的需要.
相关概念视频
01:27Language and Cognition
342
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
342
01:17Cognitivism
1.4K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.4K
01:12Observational Learning
166
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
166
01:20Introduction to Cognitive Psychology
469
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
469
01:21Cognitive Learning
237
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
237
01:22Language Development
357
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
357