Occlusion-Aware Instance Segmentation Via BiLayer Network Architectures.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Instance segmentation of highly overlapping objects is a significant challenge in computer vision.
• Existing methods struggle to differentiate object contours from occlusion boundaries.
• Occlusion relationships are not explicitly modeled in traditional instance segmentation approaches.

Purpose of the Study:

• To develop a novel approach for segmenting highly overlapping image objects.
• To explicitly model and decouple the interactions between occluding and occluded objects.
• To improve the accuracy and robustness of instance segmentation, particularly in occlusion-heavy scenes.

Main Methods:

• Proposed the Bilayer Convolutional Network (BCNet), modeling image formation as two overlapping layers.
• BCNet utilizes a top layer for detecting occluders and a bottom layer for inferring occludees.
• Investigated bilayer decoupling with Fully Convolutional Networks (FCN), Graph Convolutional Networks (GCN), and Vision Transformers (ViT).

Main Results:

• BCNet demonstrated significant and consistent improvements across various instance segmentation benchmarks (COCO, KINS, COCOA, YTVIS, OVIS, BDD100 K MOTS).
• The bilayer structure effectively decouples object boundaries and accounts for inter-object interactions.
• Exceptional performance gains were observed in scenarios with heavy occlusion.

Conclusions:

• Bilayer decoupling is a generalizable and effective strategy for improving instance segmentation.
• BCNet offers a robust solution for segmenting complex scenes with significant object overlap.
• The proposed method advances the state-of-the-art in both image and video instance segmentation.