Efficient and real-time perception: a survey on end-to-end event-based object detection in autonomous driving

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Autonomous driving relies on vision sensors for safety and accessibility.
• Event-based cameras offer advantages like high dynamic range and low power consumption over traditional cameras.
• Event data's sparse and asynchronous nature poses unique processing challenges.

Purpose of the Study:

• To provide a comprehensive survey of end-to-end event-based object detection for autonomous driving.
• To cover sensing/processing hardware, datasets, and algorithms.
• To identify shortcomings in current evaluation practices for fair comparison.

Main Methods:

• Survey of existing literature on event-based vision for autonomous driving.
• Analysis of algorithms including dense, spiking, and graph-based neural networks.
• Evaluation of system-level throughput for state-of-the-art models on specific hardware and datasets.

Main Results:

• Event-based cameras present unique processing challenges requiring specialized algorithms.
• Existing models adapted from frame-based data often underperform.
• Newer, specialized algorithms show promise but require further development and validation.
• System-level throughput was evaluated on an RTX 4090 GPU using GEN1 and 1MP datasets.

Conclusions:

• There is a need for robust, specialized algorithms to fully leverage event-based camera data for autonomous driving.
• Standardized evaluation methodologies are crucial for comparing different approaches.
• Future research should focus on improving algorithm maturity, accuracy, and efficient hardware-software co-design.