Prototypes as Anchors: Tackling Unseen Noise for online continual learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Class-incremental continual learning (CIL) models struggle with adapting to evolving domains, particularly when faced with noisy data streams.
• Existing CIL methods often assume closed-set noise, an unrealistic scenario where noise only involves known classes.
• Real-world data streams can contain open-set noise, introducing unseen classes and further complicating model adaptation.

Purpose of the Study:

• To formulate and analyze both closed-set and open-set noise in the context of CIL.
• To propose a novel method, Prototypes as Anchors (PAA), capable of handling noisy labels and unknown classes during online CIL.
• To enhance model robustness and performance in dynamic, real-world learning environments.

Main Methods:

• Formulation and analysis of closed-set and open-set noise, highlighting their impact on classifiers with unseen classes.
• Introduction of Prototypes as Anchors (PAA), a replay-based method utilizing class prototypes for denoising in representation space.
• Implementation of a dual-classifier architecture with consistency checks to ensure robust learning.

Main Results:

• Demonstrated ability of PAA to effectively distinguish and mitigate the impact of unseen classes introduced by open-set noise.
• Significant improvements in model performance and robustness across diverse datasets compared to existing CIL approaches.
• Validation of PAA's effectiveness in handling noisy labels within dynamic, evolving data streams.

Conclusions:

• PAA offers a promising solution for online class-incremental continual learning in the presence of realistic noisy labels and unknown classes.
• The method's ability to learn discriminative prototypes and employ similarity-based denoising enhances model adaptability.
• PAA provides a robust framework for continual learning in dynamic, real-world applications.