Rectification-Based Knowledge Retention for Task Incremental Learning

Area of Science:

• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models face catastrophic forgetting in task incremental learning, losing prior knowledge when trained on new tasks.
• The challenge intensifies in task incremental generalized zero-shot learning, where test classes differ from training classes.

Purpose of the Study:

• To introduce a novel approach, Rectification-based Knowledge Retention (RKR), to mitigate catastrophic forgetting in task incremental learning.
• To address both non-zero-shot and zero-shot settings within task incremental learning.
• To develop both task-aware and task-agnostic versions of the proposed method.

Main Methods:

• RKR employs weight rectifications and affine transformations to adapt models to new tasks.
• The task-aware version utilizes task labels for rapid network adaptation during testing.
• The task-agnostic version predicts the task from data and adapts the network accordingly, even without explicit task labels.

Main Results:

• The proposed RKR approach demonstrates state-of-the-art performance on benchmark datasets.
• Effective results were achieved for both non-zero-shot and zero-shot task incremental learning scenarios.
• The method successfully adapts to new tasks while retaining knowledge of previous ones.

Conclusions:

• RKR effectively addresses catastrophic forgetting in task incremental learning.
• The approach offers robust solutions for both zero-shot and non-zero-shot learning settings.
• RKR provides a flexible framework adaptable with or without task label information during inference.