大型语言模型中的概念组合:在复合词中发现隐含的关系解释,并使用上下文化的词嵌入.
Marco Ciapparelli1, Calogero Zarbo2, Marco Marelli1,3
1Department of Psychology, University of Milano-Bicocca.
Cognitive science
|March 13, 2025
概括
大型语言模型 (LLM) 可以通过分析它们的隐含含义来近似复合词的内部结构. 然而,它们在新型化合物上的表现比较旧的模型弱,这表明在语义理解方面有改进的余地.
科学领域:
- 认知科学 认知科学
- 计算语言学 计算语言学
- 心理语言学 心理语言学
背景情况:
- 大型语言模型 (LLM) 被探索为人类语义处理的潜在模型.
- 概念组合对于理解复合词至关重要,是评估语义模型的关键领域.
研究的目的:
- 调查BERT-base和Llama-2-13b大型语言模型 (LLM) 捕捉复合词的隐含含义的能力.
- 评估LLM如何代表复合词中的分解和语义关系,比较现有和新形式.
主要方法:
- 利用LLM来生成复合词的上下文嵌入及其可信的转述解释.
- 分析了表述词的可信性 (人类的评分) 与词嵌入的变化之间的关系.
主要成果:
- 嵌入变化的减少与现有和新型化合物的更高的转述可信度相关.
- 与现有化合物相比,LLM在新型化合物上的表现较弱,旧的分布模型有时表现优于它们.
结论:
- LLM提供了一个有前途的计算工具,用于近似复合词的认知结构,与心理语言学理论保持一致.
- 需要进一步的研究来增强LLM的能力,以理解新的概念组合和隐含的语义关系.
更多相关视频
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.6K
08:32Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
Published on: September 5, 2019
5.6K
相关概念视频
Language and Cognition
303
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
303
Components of Language
220
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
220
Stereotype Content Model
13.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.9K
Associative Learning
273
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
273
Multicompartment Models: Overview
76
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
76
Language Development
293
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
293
