使用构造语法解释预训练语言模型对语言结构的理解.
Leonie Weissweiler1,2, Valentin Hofmann1,3, Abdullatif Köksal1,2
1Center for Information and Language Processing, LMU Munich, Munich, Germany.
Frontiers in artificial intelligence
|October 30, 2023
概括
预训练的语言模型 (PLM) 可以识别英语比较相关构造的结构. 然而,他们很难理解这些构造的含义,这表明语言知识的局限性.
科学领域:
- 认知语言学 认知语言学
- 计算语言学 计算语言学
- 自然语言处理自然语言处理.
背景情况:
- 构造语法 (CxG) 将语言视为包含构造,集成语法和语义.
- 评估预训练语言模型 (PLM) 的语言知识对于推进AI至关重要.
- 英语比较关联词 (CC) 是一个复杂的构造,需要语法和语义上的理解.
研究的目的:
- 评估最先进的PLM处理英语比较关联词 (CC) 的能力.
- 调查建筑语法原则与PLM能力的兼容性.
- 确定PLM是否理解语法结构与CCs的语义含义.
主要方法:
- 实验使用BERT,RoBERTa和DeBERTa作为PLM的例子进行.
- 一个语法探测器评估了模型在分类CC结构中的准确性.
- 一个语义应用任务评估了模型对CC含义的理解.
主要成果:
- 所有测试的PLM (BERT,RoBERTa,DeBERTa,OPT) 都成功地识别了CC的语法结构.
- 调查的PLM在理解CC的语义意义方面表现出显著的局限性.
- 在语法分类上的表现与语义应用中的失败形成鲜明对比.
结论:
- PLM在识别语法结构方面表现出熟练,但缺乏深刻的语义理解.
- 目前的PLM在核心语言知识方面存在重大缺陷,尽管在NLP任务方面取得了进展.
- 需要进一步的研究来弥合 PLM 中的语法处理和语义理解之间的差距.
更多相关视频
相关概念视频
Components of Language
293
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
293
Language Development
378
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
378
Language and Cognition
354
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
354
Higher Mental Functions of the Brain: Language
888
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
888
Concepts and Prototypes
155
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
155
Language
232
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
232


