提高一致性和减轻偏差:用于增量学习的数据重复方法.
Chenyang Wang1, Junjun Jiang1, Xingyu Hu1
1School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China.
概括
这项研究引入了与偏差分类器 (CwD) 的一致性增强数据重复,以应对深度学习中的灾难性遗忘. 通过减少数据不一致性和平衡类权重,CwD改善了持续学习.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 深度学习模型遭受灾难性的遗忘,当连续训练.
- 现有的重播方法需要额外的内存或面临隐私问题.
- 无数据重复方法反转样本,但与真实数据产生不一致.
研究的目的:
- 解决持续学习的无数据重复方法中的数据不一致性.
- 开发一种新的损失函数和规范化技术,以提高持续学习表现.
- 引入一种名为Consistency-enhanced数据重复与偏差分类器 (CwD) 的新方法.
主要方法:
- 量化测量反转数据和真实数据之间的数据一致性.
- 开发一种新的损失函数,尽量减少数据分布之间的KL差异.
- 提出一个规范化术语,以平衡类重量,以便更好地区分.
- 使用偏差分类器 (CwD) 实现一致性增强的数据重复.
主要成果:
- 拟议的损失函数有效地减少了反向和真实数据之间的不一致性.
- 正规化术语平衡了类重量,提高了旧类样本的区分能力.
- 在CIFAR-100,Tiny-ImageNet和ImageNet100等基准数据集上,CwD的表现始终优于之前的方法.
结论:
- CwD提供了一个强大的解决方案,以灾难性的忘记在课堂增量学习.
- 该方法通过解决数据不一致和类不平衡来增强持续学习.
- CwD显示出显著的性能改善,使其成为顺序学习任务的有希望的方法.
相关概念视频
Associative Learning
288
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
288
Randomized Experiments
6.7K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.7K
Reinforcement Schedules
130
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
130
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
3.0K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
3.0K
Improving Translational Accuracy
8.8K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
8.8K
Hindsight Biases
3.4K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
3.4K


