Part-Based Semantic Transform for Few-Shot Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Few-shot semantic segmentation faces challenges due to semantic misalignment between objects.
• Existing methods struggle to effectively handle variations in object appearance and pose.

Purpose of the Study:

• To introduce a novel method, Part-based Semantic Transform (PST), for improved few-shot semantic segmentation.
• To align object semantics between support and query images by decomposing and matching object parts.

Main Methods:

• Semantic decomposition using prototype mixture models (PMMs) with an expectation-maximization (EM) algorithm.
• Semantic matching of object parts via a min-cost flow module to ensure correct correspondence.
• PST enhances network tolerance to object variations and facilitates semantic activation in query images.

Main Results:

• PST significantly improves performance over state-of-the-art methods on Pascal VOC and MS-COCO datasets.
• Achieved up to 7.79% performance improvement in five-shot semantic segmentation on MS-COCO.
• Demonstrated a moderate trade-off in inference speed and model size.

Conclusions:

• PST effectively addresses semantic misalignment in few-shot semantic segmentation.
• The proposed semantic decomposition-and-match approach offers a robust solution for handling object variations.
• PST represents a significant advancement in few-shot semantic segmentation research.