Learning Guided Convolutional Network for Depth Completion.

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Autonomous driving and robotics require dense depth perception, but LiDAR sensors provide sparse data.
• Existing neural networks often use naive fusion methods (concatenation, addition) for LiDAR-RGB data completion.
• This limits the effectiveness of multi-modal feature fusion for depth estimation.

Purpose of the Study:

• To develop a novel guided network for accurate depth completion from sparse LiDAR and RGB data.
• To improve multi-modal feature fusion by generating content-dependent, spatially-variant kernels.
• To address computational and memory constraints associated with dynamic kernel generation.

Main Methods:

• A guided network predicts kernel weights from guidance RGB images, inspired by guided image filtering.
• Predicted kernels extract depth features, enabling content-dependent, spatially-variant fusion.
• Convolution factorization is employed to reduce GPU memory and computation overhead, facilitating multi-stage fusion.

Main Results:

• The proposed method achieves state-of-the-art performance on the NYUv2 dataset.
• It ranks first on the KITTI depth completion benchmark.
• Demonstrates strong generalization across diverse datasets, lighting, weather, and point densities.

Conclusions:

• The novel guided network effectively completes sparse LiDAR data using RGB guidance.
• The method offers significant improvements over existing techniques for depth completion.
• The approach shows robust performance and generalization capabilities for real-world applications.