HiCL:无监督句子嵌入的层次对比学习
Zhuofeng Wu1, Chaowei Xiao2, Vg Vinod Vydiswaran1
1University of Michigan, Ann Arbor.
概括
本研究介绍了HiCL,这是一种分层的对比学习框架,通过考虑本地和全球关系来改善文本表示. HiCL提高了语义文本相似性 (STS) 任务的培训效率和有效性.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 传统的序列编码方法往往忽略了本地文本特征,阻碍了对较短文本的概括.
- 现有的方法在平衡培训效率和代表性学习的有效性方面面临挑战.
研究的目的:
- 提出HiCL,一个新的等级对比学习框架.
- 提高文本表示学习的培训效率和有效性.
- 为了提高语义文本相似性 (STS) 任务的性能.
主要方法:
- HiCL采用分层方法,在本地段级和全球序列级处理文本.
- 它使用对比式学习来对分段和序列表示.
- 通过首先处理短段,然后将它们汇总起来来实现高效的编码,从而解决了变压器的二次复杂性.
主要成果:
- 在七个STS任务上,HiCL显著提高了SNCSE模型的性能.
- 在BERTlarge和RoBERTalarge的平均改善率为+0.2%和+0.44%.
- 与传统方法相比,该框架显示出更高的有效性和效率.
结论:
- HiCL提供了一种有效和高效的方法来学习文本表示.
- 层次化的对比学习策略成功地模拟了本地和全球文本关系.
- 这一框架为推进语义文本相似性研究提供了坚实的基础.
相关概念视频
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Observational Learning
782
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
782
Purposive Learning
411
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
411
Cognitive Learning
960
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
960
Stereotype Content Model
15.3K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.3K
Per-Unit Sequence Models
400
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
400

