General-Purpose Deep Learning Detection and Segmentation Models for Images from a Lidar-Based Camera Sensor.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Deep learning (DL) has revolutionized robotic perception, particularly for vision sensors.
• Current autonomous systems heavily rely on DL for situational awareness.

Purpose of the Study:

• To explore the use of general-purpose DL perception algorithms on lidar-generated images.
• To assess the feasibility of using DL models designed for visual cameras on lidar data.

Main Methods:

• Focused on low-resolution, 360° field-of-view images from lidar sensors, encoding depth, reflectivity, or near-infrared data.
• Applied general-purpose DL detection and segmentation neural networks to these lidar images after preprocessing.
• Evaluated various neural network architectures qualitatively and quantitatively.

Main Results:

• Demonstrated that DL models can effectively process lidar-derived images with appropriate preprocessing.
• Showcased the potential for using these lidar images in environments where traditional vision sensors fail.
• Provided a performance analysis of different DL architectures on this novel data type.

Conclusions:

• General-purpose DL models are viable for processing lidar sensor images.
• This approach offers advantages over point cloud processing due to the maturity and availability of visual DL models.
• Opens new possibilities for robust robotic perception in diverse environmental conditions.