Real-Time LiDAR 3D Semantic Segmentation via Multi-View and Cross-Modal Compact Featuring Two-Branch Knowledge...

Area of Science:

• Computer Vision
• Robotics
• Environmental Science

Background:

• Simultaneous online mapping and semantic segmentation are crucial for environmental inspection.
• Combining visual and LiDAR data enhances segmentation but faces real-time and robustness challenges.
• Direct fusion of multi-modal and multi-view features is computationally intensive.

Purpose of the Study:

• To develop a robust and efficient method for runtime LiDAR-only semantic segmentation.
• To improve segmentation accuracy by leveraging multi-view and cross-modal information.
• To enable real-time environmental inspection and measurement tasks using handheld scanners.

Main Methods:

• Proposes a multi-view and cross-modal knowledge distillation technique.
• Hierarchically compacts multi-view and cross-model priors for distillation.
• Introduces an improved data augmentation technique (PolarMix) for realistic point cloud rendering.

Main Results:

• Achieves superior mean Intersection over Union (mIoU) compared to state-of-the-art knowledge distillation methods on SemanticKITTI and nuScenes datasets.
• Demonstrates improved segmentation accuracy and real-time performance in handheld scanner mapping experiments.
• Validates the effectiveness of the proposed knowledge distillation and data augmentation strategies.

Conclusions:

• The proposed method offers an effective solution for real-time LiDAR-only semantic segmentation.
• Knowledge distillation successfully transfers multi-view and cross-modal insights to a LiDAR-only network.
• The approach enhances the practicality of handheld scanners for environmental inspection and measurement.