Weakly-Supervised RGBD Video Object Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• RGBD video object segmentation (VOS) leverages depth data for enhanced accuracy and robustness.
• The RGBD VOS task remains underexplored due to challenges in data collection and annotation.

Purpose of the Study:

• Introduce a new benchmark, DepthVOS, for RGBD VOS.
• Propose FusedCDNet, a novel baseline model for weakly-supervised RGBD VOS.
• Address the high cost of annotation and complexity of scenes in RGBD VOS.

Main Methods:

• Developed the DepthVOS benchmark with 350 videos (55k+ frames), annotated with masks and bounding boxes.
• Proposed Fused Color-Depth Network (FusedCDNet) with a cross-modal fusion module.
• Implemented a weakly-supervised training strategy using only bounding boxes and a bounding box guideline in the first frame for inference.

Main Results:

• FusedCDNet demonstrates performance on par with top fully-supervised algorithms.
• The proposed weakly-supervised approach significantly reduces annotation burden.
• The cross-modal fusion effectively handles complex scenes.

Conclusions:

• The DepthVOS benchmark and FusedCDNet provide a strong foundation for advancing RGBD VOS research.
• Weakly-supervised methods are viable and effective for RGBD VOS.
• Open-sourcing the project aims to foster further development in the field.