Enhancing Continual Semantic Segmentation via Uncertainty and Class Balance Re-Weighting

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Continual Semantic Segmentation (CSS) aims to learn new categories without forgetting old ones.
• Pseudo-labels generated by old models are crucial but can cause forgetting if erroneous.
• Class imbalance in CSS exacerbates forgetting and confusion, extending beyond new vs. old categories.

Purpose of the Study:

• To address the impact of erroneous pseudo-labels on model forgetting in CSS.
• To mitigate confusion caused by class imbalance in continual learning.
• To propose a novel approach for enhancing CSS performance by tackling these overlooked issues.

Main Methods:

• Introduced an Uncertainty and Class Balance Re-weighting (UCB) approach.
• UCB assigns higher weights to pixels with low pseudo-label uncertainty.
• UCB also prioritizes categories with smaller proportions to address class imbalance.

Main Results:

• The UCB approach effectively reduces model forgetting in continual semantic segmentation.
• It dynamically balances category weights based on dataset characteristics.
• Experiments show improved performance across three state-of-the-art methods on Pascal-VOC and ADE20K datasets.

Conclusions:

• The proposed UCB method is simple, effective, and broadly applicable to pseudo-label-based CSS techniques.
• It enhances the learning from critical pixels, leading to better retention of old categories.
• UCB offers a robust solution for improving continual semantic segmentation models facing pseudo-label errors and class imbalance.