基于相似性的原型重建和特征重组用于非示例类增量学习的非示例类增量学习
Chao Zhou1, Jun Sun1, Vasile Palade2
1Jiangnan University, Department of Computer Science and Technology, No. 1800 Lihu Avenue, Wuxi, 214122, Jiangsu, China.
概括
这项研究引入了一种新的方法,用于在增量学习过程中对深度学习模型中的灾难性遗忘进行打击. 基于相似性的原型重建和特征重组 (SPRR) 方法有效地利用原型来保存旧知识,同时学习新信息.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 深度学习 (DL) 模型在逐步训练时面临灾难性的遗忘.
- 非实例类增量学习 (NECIL) 通过使用类原型而不是原始数据来缓解这一问题.
- 有效利用这些原型对于防止NECIL的知识损失至关重要.
研究的目的:
- 为非示例类增量学习 (NECIL) 提出一种新的方法,以解决灾难性遗忘.
- 增强对记忆类原型的利用,以提高模型稳定性和性能.
- 在增量更新期间保持以前任务的决策边界.
主要方法:
- 开发了一种基于相似性的原型重建和特征重组 (SPRR) 方法.
- 实施了特征重组机制,以不断调整原型以适应不断变化的特征空间.
- 引入了基于相似性的原型重建,以利用新数据重建旧数据特征.
- 纳入了一个知识整合策略,用于分类员培训,以与以前的特征空间保持一致.
主要成果:
- 在非模范类增量学习 (NECIL) 中,SPRR方法证明了其有效性.
- 在基准数据集上验证了性能:CIFAR-100,TinyImageNet和ImageNet-Sub.
- 实验结果证实了该方法能够缓解灾难性遗忘的能力.
结论:
- 拟议的SPRR方法通过有效管理原型,为NECIL提供了强大的解决方案.
- 该方法在增量学习场景中增强了模型稳定性和性能.
- 在不需要存储原始数据样本的情况下,SPRR成功地减轻了灾难性遗忘.
相关概念视频
01:24Concepts and Prototypes
228
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
228
02:13The Representativeness Heuristic
16.2K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
16.2K
01:24Generalization, Discrimination, and Extinction
813
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
813
01:27Associative Learning
593
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
593
01:12Observational Learning
317
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
317
01:12Modeling and Similitude
335
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
335