GeometryFormer: Semi-Convolutional Transformer Integrated with Geometric Perception for Depth Completion in...

Area of Science:

• Computer Vision
• Robotics
• Autonomous Driving

Background:

• Depth completion is crucial for Simultaneous Localization and Mapping (SLAM) and Structure from Motion (SfM) in autonomous driving.
• Vision Transformer (ViT) and convolution fusion methods have advanced depth completion accuracy.
• Existing methods struggle with detail recovery and incomplete fusion in complex scenes.

Purpose of the Study:

• To enhance depth completion accuracy by addressing limitations in detail recovery and feature fusion.
• To improve the perception of geometric structures, edges, and transparent areas in depth maps.
• To achieve state-of-the-art performance in depth completion tasks.

Main Methods:

• Proposed a semi-convolutional vision transformer to optimize local continuity.
• Designed a geometric perception module for learning spatial correlations and geometric features.
• Introduced a novel double-stage fusion strategy with learnable confidence for improved feature integration.

Main Results:

• Achieved state-of-the-art (SoTA) performance on the NYU-Depth-v2 and KITTI Depth Completion datasets.
• Attained a record low Root Mean Square Error (RMSE) of 87.9 mm on the NYU-Depth-v2 dataset.
• Demonstrated generalization ability in real-world road scenes.

Conclusions:

• The proposed method effectively enhances depth completion by optimizing local continuity and geometric perception.
• The double-stage fusion strategy significantly improves feature integration, reducing outliers and ripples.
• The model represents a significant advancement in depth completion for autonomous driving applications.