Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes

Area of Science:

• Computer Vision
• Machine Learning
• 3D Sensing

Background:

• Depth sensing technology has advanced, enabling applications like virtual reality and 3D reconstruction.
• Current depth sensors produce lower-resolution depth maps compared to color images, limiting application quality.
• Existing deep learning methods for depth super-resolution often struggle with fixed integer scale factors and incomplete depth data.

Purpose of the Study:

• To develop a novel deep learning network for simultaneous depth completion and super-resolution.
• To enable arbitrary scale factors for depth map upscaling.
• To address missing or misestimated depth data in raw depth maps.

Main Methods:

• A novel deep learning network architecture is proposed.
• The network is designed for both depth completion and super-resolution.
• The method handles arbitrary scale factors, unlike prior fixed-scale approaches.

Main Results:

• The proposed method demonstrates superior performance in depth completion and super-resolution.
• Experimental results on Middlebury stereo, NYUv2, and Matterport3D datasets validate the approach.
• The network effectively handles missing and misestimated depth data.

Conclusions:

• The novel deep learning network offers a significant advancement in depth map processing.
• The method provides high-quality depth maps with improved resolution and completeness.
• This work addresses key limitations in current depth sensing applications.