Efficient 3D Scene Semantic Segmentation via Active Learning on Rendered 2D Images.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Scene Understanding

Background:

• Semantic segmentation of large-scale 3D scenes is computationally intensive and requires extensive annotations.
• Existing methods struggle with efficiency and the need for detailed 3D data labeling.

Purpose of the Study:

• To develop a label-efficient framework for 3D scene semantic segmentation.
• To leverage 2D image analysis and active learning for accurate 3D scene understanding.

Main Methods:

• A novel framework combining Active Learning and 2D-3D semantic fusion.
• Rendering perspective 2D images from 3D scenes.
• Iterative fine-tuning of a pre-trained semantic segmentation network.
• Projecting dense 2D predictions to the 3D model for fusion and refinement.

Main Results:

• The framework achieves efficient semantic segmentation of large-scale 3D scenes.
• Demonstrated effectiveness on three large-scale indoor and outdoor datasets.
• Outperformed other state-of-the-art methods in label-efficient 3D segmentation.

Conclusions:

• The proposed rendering-segmentation-fusion iterative process effectively generates challenging samples for training.
• This approach significantly reduces the need for complex 3D annotations.
• Enables label-efficient and accurate 3D scene semantic segmentation.