SAPNet++: Evolving Point-Prompted Instance Segmentation With Semantic and Spatial Awareness.

Area of Science:

• Computer Vision
• Machine Learning
• Image Segmentation

Background:

• Single-point annotation is a cost-effective labeling method for visual tasks.
• Point-prompted instance segmentation (PPIS) faces challenges like granularity ambiguity and boundary uncertainty due to limited annotation.
• Existing PPIS methods struggle with ambiguity and imprecise mask generation.

Purpose of the Study:

• To develop a robust network for precise instance segmentation using single-point prompts.
• To overcome the limitations of granularity ambiguity and boundary uncertainty inherent in point annotations.
• To enhance the performance of point-prompted instance segmentation (PPIS).

Main Methods:

• Proposed the Semantic-Aware Point-Prompted Instance Segmentation Network (SAPNet).
• Integrated Point Distance Guidance and Box Mining Strategy to resolve granularity ambiguity.
• Introduced completeness scores for spatial granularity awareness and enhanced multiple instance learning (S-MIL).
• Employed Multi-level Affinity Refinement for pixel and semantic clue integration to reduce boundary uncertainty.

Main Results:

• SAPNet++ significantly mitigates granularity ambiguity and boundary uncertainty.
• The proposed methods demonstrate substantial improvements in segmentation performance.
• Experiments on four datasets validate the effectiveness of the developed modules.

Conclusions:

• SAPNet++ advances the state-of-the-art in point-prompted instance segmentation.
• The developed techniques offer a promising direction for efficient and accurate instance segmentation.
• This work highlights the potential of addressing annotation ambiguity for improved AI model training.