Modeling the Label Distributions for Weakly-Supervised Semantic Segmentation | JoVE Visualize

Area of Science:

Computer Vision
Machine Learning
Artificial Intelligence

Background:

Weakly-Supervised Semantic Segmentation (WSSS) offers a cost-effective alternative to fully supervised methods by utilizing limited annotation data.
Current WSSS approaches often overlook the semantic relationships between different pseudo-labels, potentially limiting performance.
Leveraging feature space proximity and distribution centers can enhance the confidence and accuracy of pseudo-labels.

Purpose of the Study:

To develop a novel WSSS framework that models underlying label distributions and utilizes cross-label constraints for improved pseudo-label generation.
To introduce an Adaptive Gaussian Mixture Model (AGMM) that leverages Gaussian Mixture Models (GMMs) to capture label distributions.
To enhance the accuracy and reliability of pseudo-labels for more effective supervision in semantic segmentation tasks.

Main Methods:

The proposed Adaptive Gaussian Mixture Model (AGMM) framework models label distributions using Gaussian Mixture Models (GMMs).
It calculates feature distribution centers of pseudo-labeled pixels and constructs the GMM based on distances to these centers.
An Online Expectation-Maximization (OEM) algorithm and a novel maximization loss are employed for adaptive GMM optimization.

Main Results:

The AGMM framework effectively models label distributions and generates high-quality pseudo-labels for reliable supervision.
It demonstrates capability in handling diverse weak label types, including image-level labels, points, scribbles, blocks, and bounding-boxes.
Experiments on PASCAL, COCO, Cityscapes, and ADE20K datasets show superior performance compared to state-of-the-art methods across all settings.

Conclusions:

The proposed AGMM framework provides a unified and effective approach for weakly-supervised semantic segmentation.
By modeling label distributions and enforcing cross-label constraints, it significantly improves the quality of pseudo-labels.
The method achieves state-of-the-art results, highlighting its potential for reducing annotation costs in semantic segmentation.