Learning Steering Kernels for Guided Depth Completion.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Reconstruction

Background:

• Guided depth completion aims to generate dense depth maps from sparse measurements and RGB images.
• Effectively utilizing extremely sparse depth data (less than 1% of pixels) remains a significant challenge.

Purpose of the Study:

• To propose a fully differentiable model for guided depth completion that excels with very sparse input data.
• To improve depth interpolation and refinement using learned kernels and residual prediction.

Main Methods:

• A novel differentiable kernel regression layer for interpolating sparse depth measurements using learned kernels.
• A residual depth refinement layer for enhancing the interpolated depth map.
• Joint learning of depth interpolation and refinement within a fully differentiable framework.

Main Results:

• The proposed method enables end-to-end training from very sparse measurements using standard convolutional neural networks.
• Differentiable kernel regression outperforms existing heuristic methods for depth interpolation.
• The approach achieves state-of-the-art performance on NYUv2 and KITTI datasets.

Conclusions:

• The developed differentiable model effectively addresses the challenge of guided depth completion with extremely sparse measurements.
• The method demonstrates strong performance and generalization capabilities across varying sparse data densities and statistics.
• This work advances the potential for low-cost, accurate dense depth map acquisition.