Color-Guided Depth Map Super-Resolution Using a Dual-Branch Multi-Scale Residual Network with Channel Interaction

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Depth map generation is crucial for 3D scene understanding.
• Existing methods struggle with detail preservation and artifact reduction in high-resolution (HR) depth maps.
• Low-resolution (LR) depth data often lacks fine details and accurate edge information.

Purpose of the Study:

• To develop an end-to-end deep learning architecture for generating HR depth maps.
• To improve the accuracy and detail of depth maps by effectively integrating color image information.
• To mitigate artifacts commonly introduced during depth map super-resolution.

Main Methods:

• Designed a dual-branch residual network architecture processing LR depth maps and HR color images separately.
• Employed multi-scale feature extraction, interaction, and upsampling within each branch.
• Utilized short-skip and long-skip connections to manage low-frequency information and focus on high-frequency details.
• Incorporated channel-wise feature fusion with channel attention to integrate color information and reduce artifacts.

Main Results:

• The proposed network successfully generates HR depth maps with improved accuracy.
• Channel-wise feature fusion effectively alleviates blurriness in depth map details, such as edges.
• Channel attention mechanism mitigates introduced depth artifacts, enhancing overall output quality.
• Experimental results demonstrate superior performance compared to existing depth map super-resolution methods.

Conclusions:

• The novel dual-branch residual network offers a robust solution for high-resolution depth map generation.
• The integration of multi-scale features and channel attention is key to achieving accurate and artifact-free depth maps.
• This approach significantly advances the state-of-the-art in depth map super-resolution and its applications.