A Transformer-Based Image-Guided Depth-Completion Model with Dual-Attention Fusion Module.

Area of Science:

• Computer Vision
• 3D Scene Perception
• Deep Learning

Background:

• Depth information is vital for understanding 3D scenes, but direct sensor capture yields incomplete and noisy depth maps.
• Image-guided depth completion aims to generate dense, accurate depth maps from sparse depth data using color images.

Purpose of the Study:

• To develop a novel network architecture for accurate and high-resolution image-guided depth completion.
• To leverage transformer models and attention mechanisms for enhanced depth map generation.

Main Methods:

• A dual-branch network architecture utilizing a transformer-based encoder for feature extraction.
• Serialization of image features into tokens and extraction of multi-scale pyramid features.
• A dual-attention fusion module combining spatial, channel, and cross-attention mechanisms for feature fusion.

Main Results:

• The proposed model achieves state-of-the-art performance on the NYUv2 and SUN-RGBD depth datasets.
• Ablation studies validate the effectiveness of the individual network modules.
• The approach successfully generates dense and accurate depth maps from sparse inputs.

Conclusions:

• Transformer models are highly effective for image-guided depth completion tasks.
• The proposed dual-attention fusion module significantly enhances feature fusion between color and depth information.
• The novel network architecture provides a robust solution for improving 3D scene perception through depth completion.