SNNTracker: Online High-Speed Multi-Object Tracking With Spike Camera.

Area of Science:

• Neuromorphic Engineering
• Computer Vision

Background:

• Traditional multi-object tracking (MOT) methods struggle with high-speed scenarios due to motion blur and low frame rates.
• Spike cameras offer continuous spatiotemporal data, but existing spike-based MOT algorithms have limitations in real-time performance and temporal continuity.

Purpose of the Study:

• To introduce SNNTracker, the first fully spiking neural network (SNN)-based MOT algorithm designed for spike cameras.
• To enable low-latency, high-speed, and robust multi-object tracking by directly processing spike streams.

Main Methods:

• Developed SNNTracker integrating a dynamic neural field (DNF)-based attention mechanism for detection.
• Employed a winner-take-all (WTA)-based tracking module with online spike-timing-dependent plasticity (STDP) for adaptive trajectory learning.
• Processed raw spike streams directly, avoiding intermediate image reconstruction.

Main Results:

• SNNTracker achieved MOTA scores above 96%, with some sequences reaching 100%, outperforming state-of-the-art ANN- and SNN-based methods.
• Demonstrated robust tracking under occlusions, lighting variations, and temporary object disappearances.
• Validated performance on newly constructed spike-camera MOT datasets covering diverse real-world scenarios.

Conclusions:

• SNNTracker advances neuromorphic vision for real-time perception, particularly in ultra-high-speed environments.
• Spike-driven SNNs offer significant advantages for low-latency, high-speed, and label-free multi-object tracking.
• The proposed method paves the way for more efficient and accurate tracking systems in robotics and autonomous driving.