适应性预训练模型的双重原型在课堂增量学习中
Zhiming Xu1, Suorong Yang2, Baile Xu1
1State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China; School of Artificial Intelligence, Nanjing University, China.
概括
本研究介绍了双原型网络与任务明智的适应 (DPTA) 打击灾难性忘记在课堂增量学习 (CIL) 使用预训练模型. DPTA提高了知识的保留和新任务的表现.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 课程增量学习 (CIL) 旨在顺序学习新课程,同时保持先前学习的课程的知识.
- 基于预训练模型 (PTM) 的方法是有效的,但在微调增量任务时容易发生灾难性的遗忘.
- 现有的CIL方法难以平衡新知识的获取与旧知识的保留.
研究的目的:
- 为基于PTM的CIL提出一个新的双原型网络,具有任务智能适应 (DPTA).
- 通过引入任务明智的适应和双重原型来缓解基于PTM的CIL中的灾难性遗忘.
- 提高模型在增量学习场景中的性能和知识保留能力.
主要方法:
- 为每个增量任务开发了一个适配器模块,以微调PTM.
- 引入了一个中心适应损失以促进中心聚类和类可分离的表示.
- 实现了用于测试时间适配器选择的双原型网络,并使用原始和增强原型改进了预测.
主要成果:
- 在多个基准指标中,DPTA的表现始终比最近的CIL方法高出1-5%.
- 在VTAB数据集上,与最先进的方法相比,取得了大约3%的改进.
- 证明有效的知识保留和改进的类分离性.
结论:
- 在基于PTM的CIL中,DPTA有效地解决了灾难性遗忘问题.
- 拟议的双原型网络和任务智能的适应显著提高增量学习性能.
- DPTA提供了一个有前途的解决方案,用于强大和高效的班级增量学习.
相关概念视频
Multi-input and Multi-variable systems
373
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
373
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Generalization, Discrimination, and Extinction
1.3K
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
1.3K
Observational Learning
795
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
795
Concepts and Prototypes
480
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
480
Survival Tree
374
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
374
