Joint Learning of Salient Object Detection, Depth Estimation and Contour Extraction.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Depth maps offer valuable information for salient object detection (SOD) in complex scenes.
• High-quality depth sensors are costly, limiting widespread application.
• General depth sensors yield noisy, sparse data, hindering depth-based SOD networks.

Purpose of the Study:

• To propose a novel multi-task and multi-modal filtered transformer (MMFT) network for RGB-D salient object detection (SOD).
• To address the limitations of noisy depth data and expensive sensors in SOD.
• To enable depth-free SOD during testing while leveraging depth information.

Main Methods:

• A multi-task learning framework unifying depth estimation, SOD, and contour estimation.
• A multi-modal filtered transformer (MFT) module with modality-specific filters.
• Task-aware feature learning through auxiliary tasks to purify depth information.

Main Results:

• The MMFT network significantly surpasses existing depth-based RGB-D SOD methods.
• The model accurately predicts high-quality depth maps and salient contours.
• The generated depth maps improve the performance of other RGB-D SOD methods.

Conclusions:

• The proposed MMFT network offers a robust and efficient solution for RGB-D SOD.
• Unifying complementary tasks and employing a filtered transformer enhances feature representation.
• The depth-free testing approach makes the method practical for real-world applications.