通过感觉类比测试正规多重学的表示结构 问题:从上下文词向量的洞察
Jiangtian Li1, Blair C Armstrong2
1Department of Psychology, University of Toronto.
Cognitive science
|March 14, 2024
概括
这项研究表明,像BERT这样的计算模型可以理解单词的含义是如何相关的,类似于人类如何理解正规的多语法. LRcos模型有效地分析了这些词语意义关系,提高了对心理词汇的理解.
科学领域:
- 计算语言学计算语言学
- 认知科学是一种认知科学.
- 自然语言处理自然语言处理.
背景情况:
- 正规的多语法涉及具有相关含义的单词 (例如,动物与肉类).
- 分布式语义模型,如BERT,代表基于上下文的单词含义.
- 了解这些模型如何处理正规多重体对于推进人工智能和认知理论至关重要.
研究的目的:
- 为了研究如何从变压器 (BERT) 的双向编码器表示表示正规的多态.
- 为了确定BERT嵌入是否可以支持感觉类比问题.
- 在计算模型中分析正规多面体的结构.
主要方法:
- 使用了LRcos模型,结合了后勤回归和等号相似性.
- 测试了模型回答常规多义论的感觉类比问题的能力.
- 分析了模型对不同正规关系内和跨越不同正规关系的共享结构的敏感性.
主要成果:
- 该LRcos模型成功地确定了正规多重学中的共享结构.
- 该模型揭示了一个"规律连续",共享结构在关系类型 (例如,动物/肉类,位置/组织) 之间有所不同.
- 证据表明,在不同的正规关系中共享潜在结构,而不仅仅是由于意义重叠.
结论:
- 计算模型可以捕捉正规多义词的细微方面.
- 这些发现为心理词典中的组织原则提供了计算证据.
- 该LRcos模型提供了一个研究单词含义表示的框架,并提供了10%的精度提升在感觉类比任务的潜力.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.9K
08:17A Semantic Priming Event-related Potential ERP Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder
Published on: April 12, 2018
10.6K
相关概念视频
The Representativeness Heuristic
15.8K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
15.8K
Polymer Classification: Stereospecificity
2.4K
Polymerization generates chiral centers along the entire backbone of a polymer chain. Accordingly, the stereochemistry of the substituent group has a significant effect on polymer properties. Polymers formed from monosubstituted alkene monomers feature chiral carbons at every alternate position in the polymer backbone. Relative to the predominant orientation of substituents at the adjacent chiral carbons, the polymer can exist in three different configurations: isotactic, syndiotactic, and...
2.4K
Vector Algebra: Graphical Method
12.1K
Vectors can be multiplied by scalars, added to other vectors, or subtracted from other vectors. The vector sum of two (or more) vectors is called the resultant vector or, for short, the resultant.
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
12.1K
Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
Polymer Classification: Architecture
2.7K
Polymers are classified as linear or branched on the basis of their chain architecture. The polymer chains in linear polymers have a long chain-like structure with minimal to no branching at all. Even if a polymer features large substituent groups on the monomer, which appear as branches to the skeleton, it is not considered a branched polymer. A branched polymer contains secondary polymer chains that arise from the main polymer chain. The branching occurs when the polymer growth shifts from...
2.7K
Cartesian Vector Notation
774
Cartesian vector notation is a valuable tool in mechanical engineering for representing vectors in three-dimensional space, performing vector operations such as determining the gradient, divergence, and curl, and expressing physical quantities such as the displacement, velocity, acceleration, and force. By using Cartesian vector notation, engineers can more easily analyze and solve problems in various areas of mechanical engineering, including dynamics, kinematics, and fluid mechanics. This...
774
