Prediction Calibration for Generalized Few-Shot Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Generalized Few-shot Semantic Segmentation (GFSS) addresses segmenting images into base and novel classes with limited data.
• Existing GFSS methods fuse classifier parameters, leading to bias towards base classes due to data imbalance.

Purpose of the Study:

• To propose a novel Prediction Calibration Network (PCN) to mitigate base class bias in GFSS.
• To improve the practical applicability of few-shot semantic segmentation.

Main Methods:

• Developed a PCN that fuses prediction scores from separate base and novel class classifiers.
• Introduced a Transformer-based calibration module to prevent bias in fused scores.
• Incorporated a cross-attention module using fused multi-level features for enhanced predictions.
• Designed a pixel-level tractable cross-attention module based on feature-score cross-covariance for generalizable training.

Main Results:

• The proposed PCN effectively addresses the base class bias issue in GFSS.
• Experiments on PASCAL- 5i and COCO- 20i datasets demonstrate superior performance over state-of-the-art methods.
• The Transformer-based calibration and cross-attention modules contribute to significant performance gains.

Conclusions:

• The PCN offers a more effective approach to GFSS by calibrating prediction scores.
• The method achieves state-of-the-art results, highlighting the potential of score fusion and Transformer-based calibration.
• This work advances the field of few-shot semantic segmentation towards more practical applications.