Weakly Supervised Semantic Segmentation via Box-Driven Masking and Filling Rate Shifting

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep Fully Convolutional Networks (FCNs) excel in semantic segmentation but require costly pixel-level annotations.
• Bounding boxes offer a weaker, more accessible form of supervision for segmentation tasks.
• Extracting precise segmentation from bounding boxes presents challenges due to class-level and region-level uncertainties.

Purpose of the Study:

• To develop a weakly supervised semantic segmentation method using bounding box annotations.
• To effectively utilize both class-level and region-level information from bounding boxes.
• To mitigate the impact of noisy or inaccurate bounding box-derived segmentation proposals.

Main Methods:

• A box-driven class-wise masking model (BCM) filters irrelevant regions.
• Mean filling rates are calculated from proposals to guide the model in ignoring mislabeled pixels.
• An anchor-based filling rate shifting module enables fine-grained instance-level supervision.
• An adaptive segmentation loss dynamically adjusts learning to reduce negative impacts of noisy proposals.
• A two-stage learning approach further enhances performance using high-quality proposals.

Main Results:

• Achieved 74.9% mean IoU in weakly supervised mode on PASCAL VOC 2012.
• Achieved 76.4% mean IoU in semi-supervised mode on PASCAL VOC 2012.
• Demonstrated effectiveness comparable to or exceeding state-of-the-art methods.

Conclusions:

• The proposed mixture model effectively leverages bounding box supervision for semantic segmentation.
• The method successfully addresses challenges of uncertain regions and noisy proposals.
• The approach offers a viable and high-performing alternative to pixel-level annotation-dependent methods.