VSCode-v2: Dynamic Prompt Learning for General Visual Salient and Camouflaged Object Detection With Two-Stage...

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Salient Object Detection (SOD) and Camouflaged Object Detection (COD) are crucial but distinct tasks in computer vision.
• Existing methods often use complex, task-specific architectures, limiting generalization.
• Previous work, VSCode, established a generalist model for SOD and COD tasks using VST and 2D prompts.

Purpose of the Study:

• To improve the generalization capabilities of the VSCode model for SOD and COD tasks.
• To introduce adaptive prompting and an optimized training strategy for enhanced performance.
• To enable zero-shot generalization to novel multimodal detection tasks.

Main Methods:

• Proposed VSCode-v2, featuring a Mixture of Prompt Experts (MoPE) layer for adaptive prompt generation.
• Implemented a two-stage training process: shared feature learning followed by task-specific feature learning.
• Incorporated knowledge distillation from a previous model and a contrastive learning mechanism with data augmentation.

Main Results:

• VSCode-v2 achieved balanced performance improvements across six SOD and COD tasks.
• The model demonstrated effective handling of various multimodal inputs.
• Exhibited zero-shot generalization capabilities on novel tasks like RGB-D Video SOD.

Conclusions:

• VSCode-v2 represents a significant advancement in generalist object detection models.
• The proposed methods enhance adaptability and generalization for SOD and COD.
• The model shows strong potential for handling diverse and unseen multimodal detection challenges.