Learned Dynamic Guidance for Depth Image Reconstruction.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Consumer depth sensors produce low-resolution, poor-quality depth images.
• High-resolution RGB cameras can enhance depth data through statistical correlation.
• Existing methods include optimization-based and learning-based approaches for guided depth reconstruction.

Purpose of the Study:

• To introduce a generalized weighted analysis sparse representation (WASR) model for guided depth image enhancement.
• To develop dynamic stage-wise operations for improved depth reconstruction quality and speed.
• To propose and validate two novel methods, DG-RBF and DG-CNN, for learning these operations.

Main Methods:

• Developed a weighted analysis sparse representation (WASR) model as a generalized framework.
• Unfolded WASR optimization into dynamically adjusted stage-wise operations.
• Proposed DG-RBF (Gaussian RBF nonlinearity) and DG-CNN (CNN nonlinearity) for task-driven learning of operations.
• Designed network architectures inspired by the WASR objective function and trained on paired data.

Main Results:

• DG-RBF and DG-CNN achieved state-of-the-art quantitative results (RMSE).
• Demonstrated superior visual quality compared to existing methods.
• Validated effectiveness in guided depth image super-resolution and realistic depth reconstruction.

Conclusions:

• The proposed WASR model and its learning-based instantiations (DG-RBF, DG-CNN) offer a powerful approach for guided depth image enhancement.
• Dynamic guidance strategies significantly improve reconstruction performance.
• The optimization-inspired network design effectively combines prior knowledge with data-driven learning.