Multi-Resolution Alignment for Voxel Sparsity in Camera-Based 3D Semantic Scene Completion.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Camera-based 3D semantic scene completion (SSC) is crucial for autonomous driving perception.
• Existing SSC methods struggle with sparse voxel data, limiting efficiency and performance.
• Voxel sparsity is a significant challenge in real-world autonomous driving scenarios.

Purpose of the Study:

• To propose a novel Multi-Resolution Alignment (MRA) approach to mitigate voxel sparsity in camera-based 3D semantic scene completion.
• To leverage multi-resolution 3D features for auxiliary supervision, improving optimization and model accuracy.
• To enhance the geometric and semantic understanding of 3D scenes from 2D image inputs.

Main Methods:

• Developed a Multi-resolution View Transformer to project 2D image features into aligned multi-resolution 3D features.
• Introduced a Cubic Semantic Anisotropy module to identify instance-level semantic significance of voxels.
• Designed a Critical Distribution Alignment module using critical voxels for auxiliary supervision across resolutions.

Main Results:

• The MRA approach significantly outperforms state-of-the-art methods on SemanticKITTI and SSCBench-KITTI-360 datasets.
• Demonstrated effectiveness in mitigating the impact of sparse voxel labels in 3D semantic scene completion.
• Achieved improved voxel-level scene perception for autonomous driving systems.

Conclusions:

• The proposed MRA approach effectively addresses the challenge of voxel sparsity in 3D semantic scene completion.
• Multi-resolution feature alignment provides a viable strategy for enhancing scene understanding in data-scarce scenarios.
• The method offers a robust foundation for perception systems in autonomous driving.