Imbalance Mitigation for Continual Learning via Knowledge Decoupling and Dual Enhanced Contrastive Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Continual learning (CL) enables models to learn from data streams without forgetting past knowledge.
• Catastrophic forgetting, where performance on old tasks degrades after learning new ones, is a major challenge in CL.
• Existing replay-based methods suffer from data imbalance, causing information suppression and weak feature discriminability.

Purpose of the Study:

• To address catastrophic forgetting in continual learning.
• To mitigate issues of information suppression and weak feature discriminability caused by data imbalance in CL.
• To propose a novel framework for enhancing knowledge retention and feature representation in continual learning.

Main Methods:

• Developed an imbalance mitigation for CL (IMCL) framework.
• Incorporated decoupled knowledge distillation (DKD) to alleviate new task suppression of old task knowledge.
• Integrated dual enhanced contrastive learning (DECL) to improve low- and high-level feature enhancement and task discrimination.

Main Results:

• The DKD approach effectively maintains old task knowledge by decoupling model output probabilities.
• The DECL approach enhances feature representations, improving discriminability between old and new tasks.
• Experiments demonstrated promising performance across task-incremental, class-incremental, and domain-incremental learning settings.

Conclusions:

• The proposed IMCL framework effectively combats catastrophic forgetting in continual learning.
• The combination of DKD and DECL offers a robust solution to data imbalance challenges in CL.
• The method shows significant potential for various incremental learning scenarios, improving model adaptability and knowledge retention.