Semantic Segmentation and Effect Optimization of 3D Point Cloud Based on 2D Semantic Segmentation and Clustering for...

Area of Science:

• Computer Vision
• Robotics
• Geospatial Science

Background:

• Construction machinery operates in unstructured environments with complex, dynamic scenes.
• 3D semantic perception is crucial but challenged by the high cost of 3D point cloud labeling.
• Existing methods struggle with the complexity and cost associated with 3D data annotation.

Purpose of the Study:

• To develop a novel 3D semantic perception scheme for unstructured construction environments.
• To enable accurate 3D understanding using only 2D image labels, reducing reliance on 3D annotations.
• To improve the operational efficiency and safety of construction machinery through enhanced perception.

Main Methods:

• Integration of 2D image semantic segmentation with 3D point cloud clustering via perspective projection.
• Refinement of projection parameters using Particle Swarm Optimization (PSO).
• Enhancement of semantic consistency using a Kd-tree-based radius nearest neighbor (RNN) matching algorithm.

Main Results:

• A weakly supervised framework achieving accurate 3D semantic understanding without annotated 3D point clouds.
• Successful perception of 3D semantic information and reconstruction of target contours.
• Achieved a mean Pixel Accuracy (mPA) of 84.72% and a mean Intersection over Union (mIoU) of 75.85%.

Conclusions:

• The proposed scheme effectively addresses the challenges of 3D semantic perception in unstructured environments.
• Validated feasibility and effectiveness through dedicated datasets and real-world testing.
• Demonstrates a cost-effective and accurate approach for 3D semantic understanding in construction.