在大型语言模型中预测下一个句子 (不是单词):模型-大脑对齐告诉我们什么关于话语理解
Shaoyun Yu1, Chanyuan Gu1, Kexin Huang1
1Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong SAR, China.
Science advances
|May 23, 2024
概括
下一个句子预测 (NSP) 培训通过增强话语层次理解和大脑数据对齐来改进大型语言模型 (LLM). 这种方法提供了更类似人类的语言表征和对语言神经科学的洞察力.
科学领域:
- 神经科学是一个神经科学.
- 人工智能的人工智能
- 计算语言学 计算语言学
背景情况:
- 目前的大型语言模型 (LLM) 主要使用单词预测,这可能无法完全捕捉人类语言理解.
- 人类语言理解包括整合单词和句子,以实现话语层次的理解.
研究的目的:
- 通过使用下一个句子预测 (NSP) 任务来研究LLM中的话语层次理解机制.
- 探索NSP预训练如何影响模型与人类大脑数据和理解能力的对齐.
主要方法:
- 用下一个句子预测 (NSP) 任务来预训大型语言模型 (LLM).
- 将模型表示与神经成像数据进行比较,重点关注涉及语言处理的大脑区域.
- 评估模型预测人类理解表现和编码上下文信息的能力.
主要成果:
- NSP预训练增强了LLM与大脑数据的对齐,特别是在右半球和多重需求网络.
- 用NSP进行预训练的模型证明了对人类理解表现的更好的捕捉.
- 通过NSP,在LLM中能够更好地编码上下文信息.
结论:
- 纳入多样化的学习目标,如NSP,导致在LLMs中更类似于人类的表现.
- 研究LLM预训练任务的神经认知可信性可以促进语言神经科学.
- 在人工智能模型中,NSP是提高话语层次理解的宝贵任务.
更多相关视频
06:15Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism
Published on: October 3, 2018
7.7K
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.7K
相关概念视频
01:27Language and Cognition
342
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
342
01:10Higher Mental Functions of the Brain: Language
793
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
793
01:22Language Development
357
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
357