Gated subspace alignment with drift compensation for parameter-efficient Class-Incremental Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Class-Incremental Learning (CIL) enables continuous learning but struggles with catastrophic forgetting.
• Existing parameter-expansion methods face representation drift and performance degradation for older classes.
• Feature subspace evolution in CIL leads to challenges in maintaining recognition accuracy for previously learned categories.

Purpose of the Study:

• To propose an efficient Class-Incremental Learning (CIL) method to address catastrophic forgetting and representation drift.
• To develop a novel approach that preserves knowledge of old classes while learning new ones effectively.
• To enhance the recognition performance and stability of models in dynamic learning environments.

Main Methods:

• Introduced Dynamic Gated Adapter for Subspace Alignment (DGASA), a lightweight method using adapters with attention-based gating.
• Utilized a frozen pre-trained backbone to construct task-specific subspaces.
• Implemented dynamic fusion of cross-task information via attention mechanisms and learned linear mapping for subspace alignment.
• Ensured alignment of old class prototypes in the current subspace without requiring access to past data.

Main Results:

• DGASA significantly improved classification accuracy across multiple benchmark datasets.
• The method demonstrated enhanced resistance to catastrophic forgetting compared to existing approaches.
• Experiments confirmed strong generalization capabilities and computational efficiency of the proposed DGASA method.
• Subspace alignment effectively mitigated representation drift and degradation of old class recognition.

Conclusions:

• DGASA offers an effective and efficient solution for Class-Incremental Learning.
• The proposed method successfully balances learning new information with preserving existing knowledge.
• DGASA provides a robust framework for continuous learning systems demanding high accuracy and stability.