SAB Net: A Semantic Attention Boosting Framework for Semantic Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Semantic segmentation models leverage contextual information for improved feature fusion.
• Existing methods face challenges in adaptively integrating nonlocal and local contextual dependencies, leading to feature imbalance and confusion.

Purpose of the Study:

• To propose an end-to-end semantic attention boosting (SAB) framework for adaptive contextual information fusion in semantic segmentation.
• To enhance segmentation accuracy by iteratively refining features across layers using semantic regularization.

Main Methods:

• Developed a pixelwise semantic attention (SAP) block to aggregate nonlocal contextual information using a pixelwise semantic metric, optimizing computation complexity.
• Introduced a categorywise semantic attention (SAC) block to balance nonlocal dependencies and local consistency with categorywise weights.
• Proposed the SAB module, integrating SAC and SAP blocks, for iterative refinement of segmentation results across layers.

Main Results:

• The SAB framework adaptively fuses contextual information, reducing the semantic gap and improving structural reasoning.
• Significantly improved segmentation results were achieved on benchmark datasets: PASCAL VOC 2012, Cityscapes, PASCAL Context, and ADE20K.
• Demonstrated superior performance compared to existing methods in quantitative evaluations.

Conclusions:

• The proposed SAB framework effectively enhances semantic segmentation by adaptively fusing contextual information iteratively.
• The novel SAP and SAC blocks address computational complexity and feature imbalance issues, leading to more robust segmentation.
• The method shows strong generalization capabilities, achieving state-of-the-art results on diverse and challenging datasets.