Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

Area of Science:

• Neuromorphic Engineering
• Computer Vision
• Hardware Acceleration

Background:

• Neuromorphic vision sensors, or Dynamic Vision Sensors (DVS), mimic the mammalian retina to capture luminosity changes with high temporal resolution.
• These event streams allow for the extraction of complex spatio-temporal patterns from visual scenes.
• The Hierarchy Of Time-Surfaces (HOTS) algorithm organizes time-surfaces to extract hierarchical features, similar to deep learning approaches.

Purpose of the Study:

• To introduce a novel Field-Programmable Gate Array (FPGA) architecture for accelerating HOTS networks.
• To enable low-power, low-latency embedded applications utilizing neuromorphic vision data.
• To evaluate the performance and accuracy of the proposed hardware architecture.

Main Methods:

• Developed a novel FPGA architecture leveraging block-RAM memory and the non-restoring square root algorithm.
• Implemented the architecture on a Zynq 7100 platform operating at 100 MHz.
• Tested the system with a gesture recognition dataset to assess accuracy and performance.

Main Results:

• Achieved latencies between 1 μs and 6.7 μs.
• Demonstrated a maximum dynamic power consumption of 77 mW.
• Obtained an accuracy loss of only 1.2% for 16-bit precision compared to the software-based HOTS algorithm.

Conclusions:

• The proposed FPGA architecture effectively accelerates HOTS networks for neuromorphic vision.
• The design is suitable for low-power, low-latency embedded systems.
• Minimal accuracy loss is observed, validating the hardware implementation for practical applications.