Cross-Image Pixel Contrasting for Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Current image semantic segmentation methods primarily focus on local pixel dependencies within images.
• Existing approaches often overlook the global context and semantic relationships between pixels across the entire training dataset.
• This limitation hinders the comprehensive understanding of image semantics.

Purpose of the Study:

• To introduce a novel pixel-wise contrastive algorithm, PiCo, for fully supervised semantic segmentation.
• To explore a new paradigm of pixel-wise metric learning by leveraging global data context.
• To improve semantic segmentation performance by explicitly considering relationships between pixels across different images.

Main Methods:

• Proposed a pixel-wise contrastive algorithm (PiCo) inspired by unsupervised contrastive representation learning.
• Enforced similarity between pixel embeddings of the same semantic class and dissimilarity between embeddings of different classes.
• Integrated PiCo into existing semantic segmentation frameworks without adding computational overhead during testing.

Main Results:

• PiCo demonstrated consistent performance improvements across diverse semantic segmentation models (DeepLabV3, HRNet, OCRNet, SegFormer, Segmenter, MaskFormer) and backbones (MobileNet, ResNet, HRNet, MiT, ViT).
• Significant gains were observed on multiple benchmark datasets, including Cityscapes, ADE20K, PASCAL-Context, COCO-Stuff, and CamVid.
• The method effectively mines global context, addressing limitations of local-context-focused approaches.

Conclusions:

• PiCo offers a novel and effective approach to semantic segmentation by incorporating global data context through pixel-wise contrastive learning.
• The algorithm is compatible with current segmentation solutions, providing performance boosts without increased testing complexity.
• This work suggests a potential shift in the standard training paradigm for semantic segmentation, encouraging exploration of cross-image pixel relationships.