Replay Without Saving: Prototype Derivation and Distribution Rebalance for Class-Incremental Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Class-incremental semantic segmentation (CISS) enables progressive learning of new classes while retaining knowledge of old ones.
• Class imbalance is a significant challenge in CISS, causing bias towards newly learned classes due to skewed training data.
• Current CISS methods often fail to adequately address the imbalance problem, leading to performance degradation.

Purpose of the Study:

• To propose a novel CISS method, STAR, that effectively addresses the class imbalance problem.
• To develop a prototype replay strategy that reintegrates past class information without requiring additional storage.
• To introduce new loss functions that preserve old class features and improve discrimination between similar classes.

Main Methods:

• STAR method utilizes prototype replay by reintroducing missing proportions of previous classes into current training samples.
• A prototype deviation technique is developed to deduce past-class prototypes, integrating classifier and feature extractor patterns.
• Two novel loss functions, Old-Class Features Maintaining (OCFM) and Similarity-Aware Discriminative (SAD) loss, are introduced to enforce cross-task feature constraints.

Main Results:

• Experiments on Pascal VOC 2012 and ADE20 K datasets demonstrate STAR's effectiveness.
• The proposed method achieves state-of-the-art performance in class-incremental semantic segmentation.
• STAR successfully mitigates class imbalance and preserves knowledge of previously learned classes.

Conclusions:

• STAR offers a robust solution to the class imbalance challenge in CISS.
• The prototype replay and novel loss functions contribute to improved performance and knowledge retention.
• This research advances the field of incremental learning for semantic segmentation.