SipMaskv2: Enhanced Fast Image and Video Instance Segmentation.

Area of Science:

• Computer Vision
• Deep Learning
• Image and Video Processing

Background:

• Single-stage instance segmentation methods often struggle with spatially adjacent instances.
• Existing methods face challenges in correlating object detection with precise mask prediction.

Purpose of the Study:

• To develop a fast, single-stage instance segmentation method that preserves spatial information.
• To enhance the delineation of spatially adjacent instances and improve mask prediction accuracy.
• To address performance limitations in current single-stage instance segmentation techniques.

Main Methods:

• Proposed SipMask method utilizing a light-weight spatial preservation (SP) module for sub-region mask predictions.
• Introduced mask alignment weighting loss and feature alignment scheme to correlate detection and segmentation.
• Developed a sample selection scheme and an instance refinement module to overcome performance bottlenecks.

Main Results:

• Achieved state-of-the-art performance on the MS COCO image instance segmentation dataset.
• Outperformed YOLACT by 3.0% (mask AP) on MS COCO with comparable speed, demonstrating real-time capabilities.
• Obtained promising results on the YouTube-VIS video instance segmentation dataset.

Conclusions:

• SipMask offers an effective and efficient solution for both image and video instance segmentation.
• The proposed SP module and complementary techniques significantly improve the handling of complex instance relationships.
• The method achieves a strong balance between speed and accuracy, advancing the field of real-time instance segmentation.