Enhancing consistency and mitigating bias: A data replay approach for incremental learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep learning models suffer from catastrophic forgetting when trained sequentially.
• Existing replay methods require extra memory or face privacy concerns.
• Data-free replay methods invert samples but introduce inconsistencies with real data.

Purpose of the Study:

• To address data inconsistencies in data-free replay methods for continual learning.
• To develop a novel loss function and regularization technique to improve continual learning performance.
• To introduce a new approach called Consistency-enhanced data replay with a Debiased classifier (CwD).

Main Methods:

• Quantitatively measuring data consistency between inverted and real data.
• Developing a novel loss function minimizing KL divergence between data distributions.
• Proposing a regularization term to balance class weights for better distinguishability.
• Implementing Consistency-enhanced data replay with a Debiased classifier (CwD).

Main Results:

• The proposed loss function effectively reduces inconsistency between inverted and real data.
• The regularization term balances class weights, enhancing the distinguishability of old class samples.
• CwD consistently outperforms previous approaches on benchmark datasets like CIFAR-100, Tiny-ImageNet, and ImageNet100.

Conclusions:

• CwD offers a robust solution to catastrophic forgetting in class incremental learning.
• The method enhances continual learning by addressing data inconsistencies and class imbalance.
• CwD demonstrates significant performance improvements, making it a promising approach for sequential learning tasks.