Through the Looking Glass: A Dual Perspective on Weakly Supervised Few-Shot Segmentation

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Meta-learning typically uses identical network architectures for homologous support-query pairs, leading to over-semantic homogenization.
• Existing methods struggle with effectively extracting inductive biases without sacrificing semantic distinctiveness.

Purpose of the Study:

• To propose a novel homologous but heterogeneous network architecture for meta-learning.
• To enhance complementarity and preserve semantic commonality in support-query pairs.
• To reduce semantic noise and amplify unique heterogeneous semantics for improved model generalization.

Main Methods:

• Introduced heterogeneous visual aggregation (HA) modules to treat support-query pairs from dual perspectives.
• Designed a heterogeneous transfer (HT) module to reduce semantic noise and amplify unique semantics.
• Incorporated heterogeneous CLIP (HC) textual information to boost multimodal model generalization.

Main Results:

• Achieved a 13.2% improvement on Pascal-5i and a 7.9% improvement on COCO-20i in weakly-supervised few-shot semantic segmentation (WFSS).
• Utilized only 1/24 of the parameters compared to existing state-of-the-art models.
• Demonstrated the first weakly-supervised model outperforming fully supervised models under identical backbone architectures.

Conclusions:

• The proposed heterogeneous network effectively addresses over-semantic homogenization in meta-learning.
• The approach significantly improves performance in WFSS tasks with remarkable parameter efficiency.
• This work sets a new benchmark for weakly-supervised models in semantic segmentation.