Double Confidence Calibration Focused Distillation for Task-Incremental Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Task-incremental learning methods using knowledge distillation face challenges like confidence bias and knowledge loss.
• These issues hinder the balance between network stability and plasticity during incremental learning.

Purpose of the Study:

• To propose a novel method, Double Confidence Calibration Focused Distillation (DCCFD), to address confidence bias and knowledge loss in task-incremental learning.
• To enhance the balance between stability and plasticity in incremental learning models.

Main Methods:

• Introduced Intratask and Intertask Confidence Calibration (ECC) modules to mitigate network overconfidence and reduce feature representation bias.
• Developed a Focused Distillation (FD) module to alleviate knowledge loss during task increments, improving model stability without sacrificing plasticity.

Main Results:

• Experimental results on CIFAR-100, TinyImageNet, and CORE-50 datasets demonstrate the effectiveness of DCCFD.
• The proposed method achieves performance matching or exceeding state-of-the-art results in class-incremental learning.

Conclusions:

• DCCFD effectively addresses key challenges in task-incremental learning, namely confidence bias and knowledge loss.
• The method serves as a plug-and-play module, consistently enhancing existing class-incremental learning approaches.