GPU Rasterization-Based 3D LiDAR Simulation for Deep Learning.

Area of Science:

• Computer Vision
• Deep Learning
• Sensor Technology

Background:

• High-quality data and annotation are crucial for deep learning but acquiring them is challenging.
• Time-of-Flight (ToF) sensors, including LiDAR, require specific data for effective deep learning model training.
• Existing data generation methods may be time-consuming or lack compatibility with real-time rendering engines.

Purpose of the Study:

• To present a GPU-accelerated simulator for generating high-quality, perfectly labeled data for any Time-of-Flight sensor.
• To develop generic algorithms that mimic unique sensor sampling patterns.
• To reduce the data acquisition and annotation burden for deep learning applications.

Main Methods:

• Utilizing a GPU-accelerated simulator that leverages the 3D graphics pipeline for efficient data generation.
• Implementing generic algorithms to accurately replicate sensor-specific sampling patterns.
• Introducing a novel loss function that integrates partially annotated real data to bridge the simulation-reality gap.

Main Results:

• The simulator significantly decreases data generation time while maintaining compatibility with real-time rendering engines.
• Trained neural networks for denoising and semantic segmentation using simulated data validated the simulator's effectiveness.
• The novel loss function enabled learning from unlabeled classes in real data, substantially reducing annotation efforts.

Conclusions:

• The developed simulator effectively addresses the data acquisition challenges in deep learning for ToF sensors.
• The approach offers a viable solution for creating large, perfectly labeled datasets essential for training robust deep learning models.
• The integration of simulated and real data via the novel loss function enhances model performance and reduces manual annotation requirements.