人工神经网络语言模型预测人类大脑对语言的反应,即使在经过发展上现实的训练后
Eghbal A Hosseini1,2, Martin Schrimpf3,4, Yian Zhang5
1Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
Neurobiology of language (Cambridge, Mass.)
|April 22, 2024
概括
在与人类相似的数据量上训练的人工神经网络 (ANN) 捕获用于语言处理的大脑活动. 即使有有限的培训,ANN也显示出与人类fMRI反应的显著对齐.
科学领域:
- 计算神经科学是一种计算神经科学.
- 认知科学是一种认知科学.
- 人工智能的人工智能是人工智能.
背景情况:
- 人工神经网络 (ANN) 越来越多地用于模拟人类语言处理.
- 一个关键的批评是ANN与人类语言学习之间的培训数据量巨大差异.
- 了解训练数据大小对ANN预测人类大脑反应的能力的影响至关重要.
研究的目的:
- 调查训练数据的数量如何影响ANN模拟人类对句子的fMRI反应的能力.
- 确定是否有足够的可信的培训数据量足以让ANN捕获人类语言处理模式.
- 探索下一个词预测性能 (困惑性) 和ANN中的神经表示相似性之间的关系.
主要方法:
- 评估了在不同数量的数据 (100万到10亿个单词) 上训练的GPT-2模型,与人类fMRI数据集相比.
- 在不同训练阶段对9亿个代币进行训练的GPT-2模型进行了评估,以评估其预测人类fMRI反应的能力.
- 使用fMRI响应预测准确度和困惑度 (下一个词预测准确度) 测量模型性能.
主要成果:
- 在开发上可信的数据量 (大约1亿个单词) 上训练的模型在捕捉人类fMRI反应方面实现了近乎最高的性能.
- 较低的困惑性 (更好的下一个词的预测) 与ANN表示和人类fMRI数据之间的更强的对齐相关.
- 在ANN中进行足够的训练以获得高的下一个词预测性能,导致能够预测人类神经反应的表示.
结论:
- 在发展现实的训练数据量 (大约1亿个单词) 足以让ANN模拟人类对句子的fMRI反应.
- 这些发现表明,通过现实训练获得足够的下一个词预测能力的ANN可以在大脑中反映人类的句子处理.
- 这项研究通过证明有限但足够的培训数据的有效性,弥合了人工语言和人类语言学习之间的差距.
更多相关视频
11:14A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
Published on: October 4, 2015
10.9K
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.9K
相关概念视频
Language Development
358
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
358
Higher Mental Functions of the Brain: Language
804
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
804
Language and Cognition
343
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
343
Neuroplasticity
341
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
341
