CalibNet: Dual-Branch Cross-Modal Calibration for RGB-D Salient Instance Segmentation

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Salient instance segmentation is crucial for understanding complex scenes.
• Existing methods struggle with accurate feature fusion from RGB and depth data.

Purpose of the Study:

• To propose CalibNet, a novel dual-branch cross-modal feature calibration architecture for RGB-D salient instance segmentation.
• To enhance the integration of RGB and depth information for improved instance-level segmentation.

Main Methods:

• Developed CalibNet, featuring dynamic interactive kernel (DIK) and weight-sharing fusion (WSF) modules.
• Incorporated a depth similarity assessment (DSA) module to refine depth features.
• Introduced a new DSIS dataset with detailed instance-level annotations.

Main Results:

• CalibNet achieved 58.0% Average Precision (AP) with a 320x480 input size on the COME15K-E test set.
• Demonstrated superior performance compared to alternative frameworks on challenging benchmarks.
• The proposed method effectively generates instance-aware kernels and mask features through cross-modal calibration.

Conclusions:

• CalibNet offers a promising and effective approach for RGB-D salient instance segmentation.
• The architecture's cross-modal feature calibration significantly improves segmentation accuracy.
• The new DSIS dataset facilitates further research in this domain.