Salient Object Detection in RGB-D Videos.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• RGB-D videos are increasingly common, yet salient object detection (SOD) in this domain remains under-explored.
• Existing research often studies RGB-D SOD and video SOD (VSOD) in isolation, lacking integrated approaches.

Purpose of the Study:

• To address the gap in RGB-D video salient object detection.
• To introduce a novel dataset (RDVS) and a sophisticated model (DCTNet+) for this task.

Main Methods:

• Construction of the RDVS dataset: a diverse RGB-D VSOD dataset with realistic depth and frame-by-frame annotations.
• Development of DCTNet+: a three-stream network emphasizing RGB, using depth and optical flow as auxiliary inputs.
• Introduction of Multi-Modal Attention Module (MAM) and Refinement Fusion Module (RFM) with Universal Interaction Module (UIM) and Holistic Multi-Modal Attentive Paths (HMAPs) for feature fusion.

Main Results:

• DCTNet+ demonstrated superior performance against 19 VSOD and 14 RGB-D SOD models on both pseudo and the proposed RDVS datasets.
• Ablation studies confirmed the effectiveness of individual modules (MAM, RFM, UIM, HMAPs) and the necessity of realistic depth data.

Conclusions:

• The proposed RDVS dataset and DCTNet+ model significantly advance the field of RGB-D video salient object detection.
• Integrating realistic depth information is crucial for improving VSOD performance.