Complementary Calibration: Boosting General Continual Learning With Collaborative Distillation and Self-Supervision.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• General Continual Learning (GCL) trains models on sequential data without task boundaries, facing catastrophic forgetting.
• Catastrophic forgetting is linked to relation and feature deviations in model learning.

Purpose of the Study:

• To propose a novel framework, Complementary Calibration (CoCa), to mitigate catastrophic forgetting in GCL.
• To address relation deviation using collaborative distillation and feature deviation via collaborative self-supervision.

Main Methods:

• Developed a collaborative distillation approach using ensemble dark knowledge to maintain old task performance and class relationships.
• Implemented collaborative self-supervision with pretext tasks and contrastive learning for discriminative feature learning.

Main Results:

• The CoCa framework demonstrated superior performance compared to state-of-the-art methods across six benchmark datasets.
• Effectively alleviated both relation and feature deviations, key challenges in GCL.

Conclusions:

• CoCa offers an effective solution for GCL by tackling core issues of catastrophic forgetting.
• The proposed methods show significant improvements in continual learning scenarios.