Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Data Analysis

Background:

• Weakly supervised semantic segmentation for point clouds aims to reduce labeling effort.
• Existing methods often ignore valuable complementary information from images in LiDAR scenarios.
• Self-training and pseudo-labeling are common but can overlook cross-modal data.

Purpose of the Study:

• To develop a novel cross-modality weakly supervised method for 3D semantic segmentation.
• To leverage unlabeled image data to enhance point cloud segmentation performance.
• To achieve high accuracy with minimal labeled data, approaching fully supervised performance.

Main Methods:

• A dual-branch network with an active labeling strategy for 2D-to-3D knowledge transfer.
• A cross-modal self-training framework with iterative parameter updating and pseudo-label estimation.
• Cross-modal association learning using cycle consistency between 3D points and 2D superpixels.
• A pseudolabel self-rectification mechanism to filter noisy labels during training.

Main Results:

• The proposed method effectively utilizes unlabeled image data for 3D point cloud segmentation.
• Cross-modal learning significantly enhances the mining of supervision from complementary data.
• The pseudolabel rectification mechanism improves label accuracy and network training.
• Experimental results show the method outperforms state-of-the-art fully supervised approaches with <1% annotations.

Conclusions:

• Integrating unlabeled image data with weakly supervised point cloud segmentation is highly effective.
• The proposed cross-modality approach offers a powerful solution for efficient 3D semantic segmentation.
• This method achieves superior performance compared to fully supervised techniques using minimal labeled data.