From Pixel to Patch: Synthesize Context-Aware Features for Zero-Shot Semantic Segmentation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Zero-shot learning (ZSL) reduces annotation burden in image classification.
• Zero-shot semantic segmentation remains underexplored despite demanding pixel-wise annotations.
• Existing methods lack effective strategies for segmenting unseen object categories.

Purpose of the Study:

• To develop a novel approach for zero-shot semantic segmentation.
• To enable the segmentation of objects from unseen categories using only semantic representations.
• To address the challenge of pixel-wise annotation in zero-shot scenarios.

Main Methods:

• Propose a context-aware feature generation network (CaGNet).
• Synthesize context-aware pixel-wise visual features for unseen categories using semantic representations and contextual information.
• Extend the approach to patch-wise feature generation and fine-tuning to incorporate inter-pixel relationships.

Main Results:

• CaGNet successfully synthesizes context-aware features for unseen categories.
• The synthesized features effectively fine-tune classifiers for segmenting unseen objects.
• Significant performance improvements over existing zero-shot semantic segmentation methods were demonstrated on benchmark datasets (Pascal-VOC, Pascal-context, COCO-stuff).

Conclusions:

• The proposed CaGNet method advances zero-shot semantic segmentation capabilities.
• Context-aware feature synthesis is effective for segmenting previously unseen object categories.
• The patch-wise extension further enhances performance by considering inter-pixel relationships.