An FPGA Accelerator for High-Speed Moving Objects Detection and Tracking With a Spike Camera

Ultra-high-speed object detection and tracking are crucial in fields such as fault detection and scientific observation. Existing solutions to this task have deficiencies in processing speeds. To deal with this difficulty, we propose a neural-inspired ultra-high-speed moving object filtering, detection, and tracking scheme, as well as a corresponding accelerator based on a high-speed spike camera. We parallelize the filtering module and divide the detection module to accelerate the algorithm and balance latency among modules for the benefit of the task-level pipeline. To be specific, a block-based parallel computation model is proposed to accelerate the filtering module, and the detection module is accelerated by a parallel connected component labeling algorithm modeling spike sparsity and spatial connectivity of moving objects with a searching tree. The hardware optimizations include processing the LIF layer with a group of multiplexers to reduce ADD operations and replacing expensive exponential operations with multiplications of preprocessed fixed-point values to increase processing speed and minimize resource consumption. We design an accelerator with the above techniques, achieving 19 times acceleration over the serial version after 25-way parallelization. A processing system for the accelerator is also implemented on the Xilinx ZCU-102 board to validate its functionality and performance. Our accelerator can process more than 20,000 spike images with 250 × 400 resolution per second with 1.618 W dynamic power consumption.