Real-Time Depth Completion With Multimodal Feature Alignment.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Depth completion is crucial for recovering dense depth maps from sparse data, often using RGB and LiDAR.
• Current multimodal fusion methods struggle with feature inconsistencies between different data types.

Purpose of the Study:

• To propose a novel feature alignment network (FANet) for enhancing multimodal feature consistency in depth completion.
• To improve the accuracy and effectiveness of depth map generation by addressing feature misalignment.

Main Methods:

• Developed a feature alignment network (FANet) with an alignment scheme to enhance consistency between RGB and LiDAR features.
• Designed an asymmetric context extraction (ACE) module to extract modality-invariant semantic contexts.
• Introduced a refinement module using residual learning for improved depth map estimation.

Main Results:

• The proposed FANet demonstrates competitive performance against real-time methods on KITTI and VOID datasets.
• The alignment scheme and refinement module show effectiveness when integrated into other depth completion methods.
• The alignment scheme incurs no additional computational cost during inference.

Conclusions:

• Feature alignment is effective in mitigating inconsistencies in multimodal fusion for depth completion.
• The proposed FANet offers an efficient and effective solution for generating accurate dense depth maps.
• The developed techniques can be broadly applied to improve existing depth completion approaches.