Balancing Learning Plasticity and Memory Stability: A parameter space strategy for class-incremental learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Continual Learning (CL) aims to enable models to learn sequentially without forgetting.
• Existing CL methods often prioritize memory stability (preventing catastrophic forgetting) over efficient new task learning.

Purpose of the Study:

• To propose a novel method, Balanced Learning Plasticity and Memory Stability (BLPMS), to concurrently enhance learning plasticity and memory stability in CL.
• To address the limitations of current CL approaches that overemphasize memory stability.

Main Methods:

• Introduced a parameter-space decomposition technique, isolating parameters into task-general and task-specific subspaces.
• Developed a training strategy that balances update rates across these subspaces to promote class-incremental learning.
• Integrated a Mixture of Experts (MoE) module with Prototypical Networks for dynamic parameter space selection during inference.

Main Results:

• BLPMS demonstrated superior performance compared to existing CL methods on multiple benchmark datasets.
• The proposed method effectively balances learning plasticity and memory stability.
• Achieved state-of-the-art results in class-incremental learning scenarios.

Conclusions:

• BLPMS offers an effective solution for the dual challenges of plasticity and stability in continual learning.
• The parameter-space decomposition and MoE-based inference strategy contribute to improved continual learning performance.
• This approach advances the field of continual learning by enabling more efficient and robust knowledge acquisition.