Joint Task-Recursive Learning for RGB-D Scene Understanding.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Monocular RGB-D scene understanding is crucial for applications like robotics and augmented reality.
• Existing methods often address tasks like depth estimation, surface normal prediction, and semantic segmentation independently.
• Jointly learning these tasks can lead to improved performance due to inter-task dependencies.

Purpose of the Study:

• To propose a novel Task-Recursive Learning (TRL) framework for joint RGB-D scene understanding.
• To enable recursive refinement of predictions through cross-task interactions.
• To achieve state-of-the-art performance on multiple scene understanding tasks.

Main Methods:

• Developed a Task-Recursive Learning (TRL) framework that recursively refines predictions.
• Introduced Task-Attentional Modules (TAM) for adaptive cross-task pattern enhancement.
• Utilized Feature-Selection Units (FS-Unit) for selective propagation of historical task experiences.
• Implemented a coarse-to-fine scale space for progressive detail refinement.

Main Results:

• The TRL framework recursively refines depth estimation, surface normal prediction, and semantic segmentation.
• Experimental results on NYU-Depth v2 and SUN RGB-D datasets show significant improvements.
• The proposed method achieves state-of-the-art performance on the evaluated benchmarks.

Conclusions:

• The TRL framework effectively integrates multiple scene understanding tasks.
• Recursive refinement and cross-task attention mechanisms are key to performance gains.
• The approach offers a promising direction for monocular RGB-D scene understanding.