宇宙启发的监督对比学习
概括
混合数据增强生成宇宙样本,在对比学习中充当硬负数. 这种灵感来自于Universe的监督对比学习 (UniCon) 改进了深度模型训练,并通过更小的批量大小实现了最先进的结果.
科学领域:
- 计算机科学 计算机科学
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 混合是一个数据增强技术,通过线性插值合成新的样本,增强模型的稳定性和概括性.
- 环球学习利用课外样本来帮助目标任务,为数据增强提供了一个新的视角.
- 监督对比学习通常需要大批次大小才能有效地学习使用硬负数的表示.
研究的目的:
- 调查Mixup在生成域内宇宙样本 (不属于任何目标类的样本) 的潜力.
- 在监督对比学习中,利用混合诱导的universeum样本作为高质量的硬负面.
- 为改进深度模型培训提出Universum启发的监督对比学习 (UniCon) 和其无监督变体 (Un-Uni).
主要方法:
- 该研究提出了UniCon,它集成Mixup生成universe负面,将它们从目标类的样本中分开.
- 该方法扩展到无监督环境中,从而产生了受无监督宇宙启发的对比模型 (Un-Uni).
- 在各种数据集上使用线性分类器评估学习表示的有效性.
主要成果:
- UniCon 在多个数据集上实现了最先进的性能,比现有方法显著改进.
- 在CIFAR-100上,UniCon达到81.7%的top-1准确度,在实质上较小的批量大小 (256比1024) 下,超过了之前的工作5.2%.
- 与CIFAR-100上最先进的方法相比,Un-Uni还表现出优越的性能.
结论:
- 混合诱导的宇宙样本是监督对比学习中的有效硬负数,减少了对大批次大小的需求.
- UniCon和Un-Uni为数据生成和对比学习提供了新的方法,实现了卓越的性能.
- 提出的方法推进了数据增强策略和深度模型培训的对比学习框架.
相关概念视频
01:12Observational Learning
213
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
213
01:18Introduction to Learning
478
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
478
01:27Associative Learning
452
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
452
01:22Purposive Learning
146
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
146
01:21Cognitive Learning
433
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
433
01:24Generalization, Discrimination, and Extinction
636
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
636