Learning From Box Annotations for Referring Image Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Referring image segmentation (RIS) traditionally relies on fully convolutional networks (FCNs).
• Existing RIS methods demand extensive pixel-level annotations, limiting their practical application.
• Bounding box (BB) annotations offer a less labor-intensive alternative for supervision.

Purpose of the Study:

• To develop a weakly supervised RIS method utilizing bounding box (BB) annotations.
• To overcome the limitations of pixel-level annotation requirements in traditional RIS.
• To improve the efficiency and accuracy of referring image segmentation.

Main Methods:

• Introduced an adversarial boundary loss in the first stage to extract object contours from BBs.
• Utilized extracted contours for generating pseudoground-truth (PGT) by selecting appropriate region proposals.
• Designed a co-training (Co-T) strategy in the second stage to iteratively refine pseudolabels and mitigate noisy label effects.

Main Results:

• The proposed method successfully generates high-quality segmentation masks.
• Achieved a processing speed of 63 frames per second on benchmark datasets.
• Demonstrated effectiveness across four standard benchmark datasets for RIS.

Conclusions:

• The developed weakly supervised RIS method effectively leverages BB annotations.
• The co-training strategy significantly purifies pseudolabels, enhancing model robustness.
• The method offers a practical and efficient solution for referring image segmentation tasks.