Towards a Meaningful 3D Map Using a 3D Lidar and a Camera.

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Geomatics Engineering

Background:

• Semantic 3D maps are crucial for robot navigation and surveying.
• Existing camera-based semantic mapping methods struggle with large-scale environments due to computational demands.
• Combining 3D Lidar and cameras offers a promising solution for efficient semantic mapping.

Purpose of the Study:

• To develop an accurate and efficient semantic 3D mapping algorithm.
• To address the limitations of previous methods in large-scale applications.
• To enhance the quality and reliability of semantic 3D maps.

Main Methods:

• Integrated GPS and IMU for system odometry estimation.
• 3D Lidar point cloud registration using estimated odometry.
• Convolutional Neural Network (CNN)-based semantic segmentation for environmental understanding.
• Developed incremental semantic labeling with coordinate alignment, error minimization, and semantic information fusion.
• Batch processing for map refinement, enhancing label distribution and removing dynamic object traces.

Main Results:

• The proposed algorithm achieves superior performance compared to state-of-the-art methods.
• Demonstrated high accuracy and intersection over union (IoU) in challenging experimental sequences.
• Successfully integrated point cloud data with semantic information for comprehensive 3D mapping.

Conclusions:

• The developed algorithm provides an effective solution for semantic 3D mapping in large-scale environments.
• The combination of Lidar, camera, GPS, and IMU, along with advanced deep learning techniques, significantly improves mapping quality.
• The approach offers a robust and accurate method for applications requiring detailed semantic 3D representations.