Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep neural networks excel in semantic segmentation but struggle with catastrophic forgetting and require extensive pixel-level annotations.
• Incremental learning in semantic segmentation is challenged by background class semantic shift due to partial annotations.

Purpose of the Study:

• To develop a novel incremental class learning approach for semantic segmentation that mitigates catastrophic forgetting and reduces annotation dependency.
• To address the semantic shift of background pixels during incremental training.

Main Methods:

• A novel incremental class learning approach for semantic segmentation is proposed.
• Revisiting the distillation paradigm with novel loss terms to account for background shift.
• Introducing a new strategy for classifier parameter initialization to prevent background bias.
• Extending the approach to weakly supervised segmentation using point- and scribble-based annotations.

Main Results:

• The proposed method effectively handles background semantic shift in incremental learning.
• The approach demonstrates improved performance in weakly supervised semantic segmentation.
• Significant outperformance over state-of-the-art methods on Pascal-VOC, ADE20K, and Cityscapes datasets.

Conclusions:

• The novel approach successfully tackles catastrophic forgetting and annotation limitations in semantic segmentation.
• The method is effective for both fully and weakly supervised incremental class learning.
• This work advances the field of semantic segmentation by providing a more robust and efficient learning strategy.