A recurrent YOLOv8-based framework for event-based object detection

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence
• Biologically Inspired Systems

Background:

• Conventional RGB sensors struggle with motion blur and extreme lighting in object detection.
• Event-based cameras offer superior performance in dynamic scenes and low-power applications.
• Integrating event-based sensors with advanced object detection is crucial for next-gen systems.

Purpose of the Study:

• To develop an object detection framework leveraging event-based cameras.
• To enhance the YOLOv8 object detection system with spatiotemporal modeling using event data.
• To introduce novel methods for efficient event data encoding and augmentation.

Main Methods:

• Introduced Recurrent YOLOv8 (ReYOLOv8), integrating recurrent layers into YOLOv8.
• Developed Volume of Ternary Event Images (VTEI) for low-latency event data encoding.
• Implemented Random Polarity Suppression (RPS) data augmentation tailored for event sensors.

Main Results:

• ReYOLOv8 achieved significant mAP improvements on GEN1 (up to 5%) and PEDRo (up to 18%) datasets.
• Reduced trainable parameters and model size while maintaining real-time processing speeds.
• Demonstrated superior performance over conventional methods in challenging visual environments.

Conclusions:

• Event-based vision sensors combined with advanced frameworks like ReYOLOv8 show significant potential.
• ReYOLOv8 effectively bridges biological vision principles with AI for robust visual processing.
• The framework enables efficient and reliable object detection in dynamic and complex environments.