用人类工具挑战大型语言模型的"智力":以意大利语对前额功能进行神经心理学调查
Riccardo Loconte1, Graziella Orrù2, Mirco Tribastone1
1Molecular Mind Lab, IMT School of Advanced Studies Lucca, Lucca, Italy.
Heliyon
|October 21, 2024
概括
像GPT-3.5和Claude2这样的大型语言模型 (LLM) 在人类智能测试中表现不一致,特别是在计划和理解意图方面. GPT-4的性能更好,但目前的AI不能完全复制人类的认知功能.
科学领域:
- 神经科学是一个神经科学.
- 人工智能的人工智能
- 认知科学 认知科学
背景情况:
- 人工智能 (AI) 研究社区使用基准来测量大型语言模型 (LLM) 的智能.
- 人类的智力与前额叶完整性有关,这对更高层次的认知至关重要.
- 从历史上看,LLM一直在努力完成需要前额功能的任务,这对复制类似人类的智能构成了挑战.
研究的目的:
- 严格调查法学士的情报水平.
- 评估GPT-3.5,GPT-4,Claude2和Llama2使用人类神经心理测试对前额功能.
- 为了确定标准化的人类认知测试是否可以挑战LLM的表现.
主要方法:
- 用意大利语进行的神经心理测试,评估GPT-3.5,GPT-4,Claude2和Llama2的前额功能.
- 与人类规范数据相比,LLM的表现.
- 评估计划能力,识别语义荒谬,以及思想理论.
主要成果:
- GPT-3.5和Claude2显示出不一致的结果,规划受损,难以识别荒谬和意图.
- 在大多数测试中,Llama2的表现很差.
- 在大多数任务中,GPT-4在正常范围内执行,除了规划.
结论:
- 这种LLM的新兴能力还没有完全模仿人类的认知功能.
- 标准化的神经心理学电池适用于评估LLM绩效.
- 与其他评估的LLM相比,GPT-4表现出更强大的前额头类能力,尽管规划仍然是一个挑战.
相关概念视频
01:27Language and Cognition
329
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
329
01:10Higher Mental Functions of the Brain: Language
764
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
764
01:17Cognitivism
1.4K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.4K
01:20Introduction to Cognitive Psychology
439
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
439
01:14Role of Cerebellum and Prefrontal Cortex in Memory
376
The cerebellum, while traditionally associated with motor control, also plays a crucial role in memory, particularly in procedural memory, which involves learning motor tasks that become automatic through repetition. For example, studies have shown that when the cerebellum is damaged, individuals or animals lose the ability to learn conditioned motor responses, such as the conditioned eye-blink response in classical conditioning experiments with rabbits. This study demonstrates the...
376