WS-SAM: Generalizing SAM to Weakly Supervised Object Detection With Category Label.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Effective object detection typically requires large, meticulously annotated datasets, which are costly and time-consuming to create.
• Weakly supervised learning methods offer reduced annotation costs but often suffer from insufficient performance due to limited supervision.
• The Segment Anything Model (SAM) shows strong generalization capabilities, inspiring its application to address data deficiencies in weakly supervised object detection.

Purpose of the Study:

• To adapt the Segment Anything Model (SAM) for weakly supervised object detection, enabling category label assignment.
• To overcome limitations of direct SAM deployment, such as the need for expert prompts and category unawareness.
• To improve the performance of weakly supervised object detection by leveraging SAM's capabilities.

Main Methods:

• Proposed WS-SAM, an adaptive prompt generator utilizing spatial and semantic information for iterative self-prompting.
• Developed a segmentation mask refinement module and formulated label assignment as a shortest path optimization problem.
• Implemented a bidirectional adapter to address domain discrepancy by incorporating domain-specific information.

Main Results:

• WS-SAM effectively generalizes the Segment Anything Model (SAM) for weakly supervised object detection with category labels.
• The adaptive prompt generator and refinement modules significantly enhance detection accuracy.
• Experimental results on PASCAL VOC and MS COCO datasets show clear improvements over state-of-the-art methods.

Conclusions:

• WS-SAM successfully compensates for insufficient supervised information in object detection tasks.
• The proposed method offers a practical and effective approach to weakly supervised object detection.
• This work demonstrates the potential of foundation models like SAM in advancing specialized AI fields.