Asynchronous Spatio-Temporal Memory Network for Continuous Event-Based Object Detection.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Event cameras provide high temporal resolution and dynamic range, offering advantages for object detection.
• Challenges remain in effectively learning spatio-temporal representations from asynchronous event data.
• Existing methods struggle to fully exploit the rich temporal information inherent in event streams.

Purpose of the Study:

• To develop a novel network architecture for object detection using asynchronous event data.
• To improve the learning of spatio-temporal representations from continuous event streams.
• To enhance object detection performance in challenging scenarios like fast motion and low light.

Main Methods:

• Proposed an asynchronous spatio-temporal memory network (ASTMNet) that directly processes asynchronous events.
• Introduced an adaptive temporal sampling strategy and temporal attention convolutional module for asynchronous attention embedding.
• Designed a spatio-temporal memory module with an inter-weaved recurrent-convolutional architecture to exploit temporal cues.

Main Results:

• ASTMNet achieved superior performance over state-of-the-art frame-based detectors on three benchmark datasets.
• Significant performance gains were observed: 7.6% on KITTI, 10.8% on Gen1, and 10.5% on 1Mpx Detection Dataset.
• Demonstrated robust object detection capabilities even in challenging conditions where conventional cameras fail.

Conclusions:

• Event cameras enable robust object detection, particularly in scenarios with fast motion and challenging lighting.
• ASTMNet effectively leverages asynchronous event data for continuous and accurate object detection.
• The proposed approach advances the utilization of event-based vision for real-world perception tasks.