在大型语言模型中重新评估思维理论评估
Jennifer Hu1,2, Felix Sosa3,4, Tomer Ullman3,4,5
1Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, USA.
概括
研究人员重新评估了大语言模型 (LLM) 如何对心智理论 (ToM) 进行测试. 他们建议专注于基础计算,而不仅仅是行为,以澄清LLM.
科学领域:
- 认知科学 认知科学
- 人工智能的人工智能
- 语言学的语言学.
背景情况:
- 关于大语言模型 (LLM) 是否表现出心智理论 (ToM) 的重大科学和公共利益.
- 关于LLM ToM能力的现有证据是混合的,评估未能达成共识.
- ToM的定义涉及到对他人的心理状态的推理.
研究的目的:
- 在大型语言模型 (LLM) 中重新评估目前的思维理论 (ToM) 评估状况.
- 通过从认知科学中汲取灵感来解决LLM ToM研究中的模两可.
- 为了澄清LLM评估是否应该关注匹配人类行为或潜在的计算过程.
主要方法:
- 受认知科学原则的启发,重新审视现有的LLM ToM评估方法.
- 分析当前的评估实践,以识别与纯 ToM 测量的偏差.
- 概念框架的发展,以区分行为模仿和计算理解在LLMs.
主要成果:
- 关于LLM ToM的分歧的一个关键原因是缺乏清晰的判断是否评估行为模仿或潜在的计算.
- 目前的评估可能不仅仅是测量ToM能力,这导致了混乱.
- 结果的差异源于对模型性能的不同期望.
结论:
- 未来的研究应该澄清LLMs中ToM的行为和计算评估之间的区别.
- 研究ToM与实用沟通之间的关系对于理解人工和人类认知至关重要.
- 推进ToM评估方法将改善我们对人工智能和人类沟通的理解.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
578
06:15Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism
Published on: October 3, 2018
7.8K
相关概念视频
Language and Cognition
441
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
441
Higher Mental Functions of the Brain: Language
1.0K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.0K
Language Development
449
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
449
Lazarus's Cognitive Appraisal Theory
526
Cognitive psychologist Richard Lazarus proposed the cognitive-mediational theory of emotions, which emphasizes how individuals' assessments of stressors significantly affect their experience of stress. According to Lazarus, the stress response is determined by a two-step appraisal process: primary appraisal and secondary appraisal. These cognitive appraisals help individuals evaluate the potential impact of a stressor and determine the adequacy of their coping resources.
Primary Appraisal:...
Primary Appraisal:...
526
Modeling in Therapy
145
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
145
Cognitive Theories: Lazarus Mediational Theory of Emotion
1.2K
Richard Lazarus' cognitive mediational theory highlights the pivotal role of cognitive appraisal in shaping emotional responses. According to this theory, the evaluation of a stimulus — based on personal values, goals, beliefs, and expectations — mediates the emotional response. This appraisal process is immediate and often occurs unconsciously, influencing the intensity and nature of the resulting emotion.
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
1.2K
