Dynamic Vision Sensor-Driven Spiking Neural Networks for Low-Power Event-Based Tracking and Recognition

Area of Science:

• Neuromorphic Engineering
• Computer Vision
• Artificial Intelligence

Background:

• Spiking Neural Networks (SNNs) offer energy-efficient processing for Dynamic Vision Sensors (DVSs).
• Challenges remain in optimizing SNN training and handling spatio-temporal complexity for real-time embedded applications like object tracking.
• Existing methods often rely on conventional Convolutional Neural Networks (CNNs), limiting efficiency for DVS data.

Purpose of the Study:

• To propose a novel, pure SNN framework, the Dynamic Tracking with Event Attention Spiking Network (DTEASN).
• To address limitations in SNN training and spatio-temporal complexity for DVS-based real-time embedded sensing.
• To bypass CNN operations and reduce GPU dependency for enhanced efficiency.

Main Methods:

• Developed an event-driven multi-scale attention mechanism and a spatio-temporal event convolver for enhanced feature extraction from DVS events.
• Introduced an Event-Weighted Spiking Loss (EW-SLoss) to prioritize informative events and improve noise robustness.
• Incorporated a lightweight event tracking mechanism and a custom synaptic connection rule for improved efficiency in low-power, edge deployments.

Main Results:

• DTEASN demonstrated superior performance on DVS object recognition and tracking benchmarks compared to conventional methods.
• Achieved improvements in accuracy, latency, event throughput, spike rate, memory footprint, and spike-efficiency.
• Showcased enhanced overall computational efficiency under typical DVS settings.

Conclusions:

• The DTEASN framework effectively processes DVS event streams using a pure SNN architecture.
• The proposed components significantly enhance spatio-temporal feature extraction and learning optimization.
• DTEASN is suitable for highly parallel neuromorphic hardware, enabling on- or near-sensor inference for embedded applications.