PMG-SAM: Boosting Auto-Segmentation of SAM with Pre-Mask Guidance.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Foundational vision models like Segment Anything Model (SAM) face challenges in fully automatic, targeted object segmentation.
• The "segment everything" mode of SAM exhibits localization blindness and computational inefficiency, hindering performance on tasks such as Dichotomous Image Segmentation (DIS).

Purpose of the Study:

• To develop an efficient and accurate framework, PMG-SAM, for automatic targeted object segmentation.
• To address the localization bottleneck in large vision models for prompt-free segmentation scenarios.

Main Methods:

• Proposed PMG-SAM framework utilizing a Pre-Mask Guided paradigm for automatic targeted segmentation.
• Employed a dual-branch encoder to generate a global Pre-Mask, serving as an internal prompt for the segmentation decoder.
• Introduced a Dense Residual Fusion Module (DRFM) for iterative, multi-scale feature co-refinement to enhance Pre-Mask quality.

Main Results:

• PMG-SAM significantly improved performance on challenging DIS and Camouflaged Object Segmentation (COS) tasks.
• Achieved a maximal F-measure of 0.815 on the DIS-TE2 benchmark, a substantial increase from SAM's 0.283.
• Outperformed SAM and SAM2 in ground-truth bounding box prompted modes, despite being fully automatic and using fewer trainable parameters (22.9 M).

Conclusions:

• PMG-SAM presents an effective and efficient solution for automatic targeted object segmentation.
• The Pre-Mask Guided paradigm successfully overcomes the localization limitations of large vision models in prompt-free settings.
• This framework offers a promising direction for advancing prompt-free segmentation capabilities in computer vision.