Towards Real Zero-Shot Camouflaged Object Segmentation Without Camouflaged Annotations.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Camouflaged Object Segmentation (COS) is hindered by limited annotated data, making pixel-level annotation difficult and expensive.
• Existing transformer models for Salient Object Segmentation (SOS) primarily use global attention, which is insufficient for COS's local and global attention biases.

Purpose of the Study:

• To develop an effective zero-shot Camouflaged Object Segmentation (COS) framework that eliminates the need for manual annotations.
• To investigate and leverage the distinct attention patterns of camouflaged objects compared to salient objects.

Main Methods:

• A framework incorporating a Masked Image Modeling (MIM) encoder with Parameter-Efficient Fine-Tuning (PEFT) for local and global feature extraction.
• Integration of a Multimodal Large Language Model (M-LLM) for semantic understanding, aligned with visual features via Multi-scale Fine-grained Alignment (MFA).
• A learnable codebook for efficient M-LLM representation during inference, reducing computational load.

Main Results:

• Achieved state-of-the-art zero-shot COS performance with $F_{\beta }^{w}$Fβw scores of 72.9% on CAMO and 71.7% on COD10K.
• Demonstrated inference speeds comparable to traditional models (18.1 FPS) after optimization.
• Showcased strong performance in polyp and underwater scene segmentation, outperforming baselines in both zero-shot and supervised settings.

Conclusions:

• The proposed framework successfully enables zero-shot COS by adapting to local biases and incorporating global semantic information.
• The method offers a computationally efficient and versatile solution for various segmentation tasks beyond COS.
• This research significantly advances the potential for automated segmentation in data-scarce scenarios.