Box2Mask: Box-Supervised Instance Segmentation via Level-Set Evolution.

Area of Science:

• Computer Vision
• Deep Learning
• Image Segmentation

Background:

• Instance segmentation typically requires pixel-wise mask annotations, which are labor-intensive to create.
• Box-supervised instance segmentation offers a more efficient alternative using simpler bounding box annotations.
• Existing box-supervised methods often struggle to achieve accuracy comparable to fully supervised approaches.

Purpose of the Study:

• To propose a novel single-shot instance segmentation approach, Box2Mask, that utilizes bounding box supervision.
• To integrate classical level-set evolution with deep neural networks for accurate mask prediction.
• To demonstrate the effectiveness of Box2Mask across diverse image datasets.

Main Methods:

• Box2Mask integrates level-set evolution into deep neural networks for mask prediction.
• It uses both input images and deep features to evolve level-set curves implicitly.
• A local consistency module mines spatial relations, and two frameworks (CNN and Transformer-based) are developed.

Main Results:

• The proposed Box2Mask approach achieves accurate mask prediction with only bounding box supervision.
• Experimental results on five challenging datasets show outstanding performance.
• With a Swin-Transformer backbone, Box2Mask reached 42.4% mask AP on COCO, matching fully mask-supervised methods.

Conclusions:

• Box2Mask effectively performs box-supervised instance segmentation, significantly reducing annotation effort.
• The integration of level-set evolution with deep learning offers a powerful new direction for instance segmentation.
• The method demonstrates broad applicability across various image domains, including general scenes, remote sensing, medical, and scene text.