Depth Restoration From RGB-D Data via Joint Adaptive Regularization and Thresholding on Manifolds

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Depth map restoration is crucial for 3D scene understanding.
• Existing methods struggle with complex depth map geometries and self-similar structures.
• RGB-D data offers rich information but requires sophisticated processing for accurate depth recovery.

Purpose of the Study:

• To propose a novel depth restoration algorithm leveraging both local and non-local manifold properties.
• To enhance the accuracy and quality of depth maps derived from RGB-D data.
• To develop a robust optimization framework for joint manifold regularization and thresholding.

Main Methods:

• Utilizing local manifold models to preserve pixel neighborhood relationships and promote smoothing.
• Employing non-local manifold characteristics to build adaptive bases for extracting image patterns and self-similar structures.
• Defining a manifold thresholding operator in 3D adaptive orthogonal spectral bases for frequency-based restoration.
• Implementing a unified alternating direction method of multipliers (ADMM) optimization framework.

Main Results:

• The proposed algorithm effectively restores depth maps by combining local and non-local geometric information.
• Manifold regularization and adaptive thresholding jointly address the inverse problem of depth recovery.
• Experimental results show superior performance over state-of-the-art methods in objective and subjective evaluations.
• The method successfully accounts for self-similar structures and complex geometry in depth maps.

Conclusions:

• The novel depth restoration algorithm offers significant improvements in depth map quality.
• Combining local and non-local manifold characteristics provides a powerful approach for depth recovery.
• The unified optimization framework effectively integrates adaptive manifold regularization and thresholding for robust performance.