PMRVT: Parallel Attention Multilayer Perceptron Recurrent Vision Transformer for Object Detection with Event Cameras

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Object detection in high-speed, dynamic environments is challenging for conventional cameras due to motion blur and latency.
• Event cameras offer an alternative with high dynamic range and low latency, but existing methods have high computational costs and limited temporal modeling.
• Real-time event-based vision applications require improved detection accuracy and efficiency.

Purpose of the Study:

• To present PMRVT (Parallel Attention Multilayer Perceptron Recurrent Vision Transformer), a unified framework for event-based object detection.
• To systematically balance early-stage efficiency, spatial expressiveness, and long-horizon temporal consistency in event-based detection.
• To achieve strong accuracy and real-time performance for dynamic environments.

Main Methods:

• Developed a hybrid hierarchical backbone for efficient processing.
• Implemented a Parallel Attention Feature Fusion (PAFF) mechanism with a coordinated dual-path design.
• Employed a temporal integration strategy for enhanced consistency.

Main Results:

• PMRVT achieved 48.7% mAP on the Gen1 dataset and 48.6% mAP on the 1 Mpx dataset.
• Inference latencies were 7.72 ms (Gen1) and 19.94 ms (1 Mpx).
• Demonstrated a 1.5 pp accuracy improvement and 8% latency reduction compared to state-of-the-art methods.

Conclusions:

• PMRVT offers a favorable balance between accuracy and speed for event-based object detection.
• The framework provides a reliable solution for real-time event-based vision applications.
• PMRVT addresses limitations of existing methods in computational cost and temporal modeling.