Multi-granularity knowledge distillation and prototype consistency regularization for class-incremental learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) suffer from catastrophic forgetting when learning new tasks sequentially.
• Existing class-incremental learning (CIL) methods rely on storing data exemplars or complex generative models, posing memory, privacy, or efficiency challenges.

Purpose of the Study:

• To propose a novel exemplar-free method for CIL that mitigates catastrophic forgetting.
• To enhance model performance without access to previous task data.

Main Methods:

• Developed a method combining multi-granularity knowledge distillation and prototype consistency regularization (MDPCR).
• Employed knowledge distillation in deep feature space, focusing on multi-scale self-attentive features, feature similarity probability, and global features.
• Introduced prototype consistency regularization (PCR) to ensure consistency between old and enhanced prototypes, improving robustness and reducing bias.

Main Results:

• MDPCR effectively alleviates catastrophic forgetting by maximizing the retention of previous knowledge.
• The method demonstrates superior performance compared to existing exemplar-free CIL approaches.
• MDPCR also outperforms typical exemplar-based CIL methods on benchmark datasets.

Conclusions:

• MDPCR offers an effective solution for CIL, overcoming limitations of data storage and generative models.
• The proposed approach significantly enhances incremental learning capabilities while preserving knowledge from previous tasks.
• This method provides a robust and efficient alternative for CIL applications.