Learning Depth with Convolutional Spatial Propagation Network.

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Depth estimation is crucial for various applications like robotics and autonomous driving.
• Existing linear propagation models, such as Spatial Propagation Networks (SPN), face limitations in speed and efficiency.
• Convolutional Neural Networks (CNNs) have shown promise in image processing tasks.

Purpose of the Study:

• To propose and evaluate the Convolutional Spatial Propagation Network (CSPN) for enhanced depth estimation.
• To demonstrate CSPN's efficiency and effectiveness compared to existing state-of-the-art methods.
• To adapt CSPN for specific depth estimation challenges like depth completion and stereo matching.

Main Methods:

• Developed CSPN, a linear propagation model utilizing recurrent convolutional operations.
• Learned pixel affinities through a deep CNN.
• Integrated CSPN with existing depth estimation networks.
• Designed specialized modules for depth completion (handling sparse samples) and stereo matching (3D convolution, spatial pyramid pooling).

Main Results:

• CSPN is 2-5x faster than SPN.
• Significant improvements in depth accuracy were achieved when CSPN was integrated into SOTA networks.
• Reduced depth error by over 30% on NYU v2 and KITTI datasets for depth completion.
• Achieved 1st rank on KITTI Stereo 2012 and 2015 benchmarks for stereo matching.

Conclusions:

• CSPN is a highly effective and efficient model for depth estimation tasks.
• The proposed adaptations of CSPN are beneficial for depth completion and stereo matching.
• CSPN represents a significant advancement in the field of depth estimation, offering both speed and accuracy gains.