Low-latency automotive vision with event cameras

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Current advanced driver assistance systems (ADAS) rely on RGB cameras, facing bandwidth-latency trade-offs.
• Event cameras offer high temporal resolution and sparsity, reducing bandwidth and latency.
• Existing event-camera algorithms struggle to match the accuracy of image-based systems or sacrifice efficiency.

Purpose of the Study:

• To develop a hybrid object detection system combining event and frame-based cameras.
• To overcome the accuracy-efficiency trade-off in current event-camera algorithms.
• To leverage the strengths of both event and RGB cameras for improved ADAS perception.

Main Methods:

• Proposed a hybrid object detector integrating event and RGB camera data.
• Exploited high temporal resolution and sparsity of event data.
• Utilized rich, low temporal resolution information from standard images.

Main Results:

• Achieved high-rate object detections with reduced perceptual and computational latency.
• Demonstrated that a 20 fps RGB camera plus an event camera matches 5,000 fps camera latency.
• Maintained accuracy comparable to high-frame-rate systems with significantly lower bandwidth (45 fps).

Conclusions:

• The hybrid approach effectively combines the benefits of event and frame-based vision.
• This method offers an efficient and robust solution for perception in ADAS.
• Unlocks the potential of event cameras for real-time, high-performance driving assistance.