Event-Based Optical Flow Estimation with Spatio-Temporal Backpropagation Trained Spiking Neural Network.

Area of Science:

• Computational Neuroscience
• Computer Vision
• Neuromorphic Engineering

Background:

• Event cameras offer advantages like low power and high dynamic range, excelling in extreme conditions and high-speed motion capture.
• Optical flow from event cameras provides crucial movement information but existing neural network methods are computationally intensive and power-hungry.
• Spike neural networks (SNNs) are compatible with event camera data and offer ultra-low power consumption on neuromorphic hardware due to sparse coding.

Purpose of the Study:

• To propose an end-to-end spike neural network for predicting optical flow from event camera data.
• To address the computational and power consumption challenges of existing optical flow prediction methods for event cameras.
• To enable efficient, low-power hardware implementation of optical flow prediction for event cameras.

Main Methods:

• Developed an end-to-end spike neural network tailored for discrete spatiotemporal data streams from event cameras.
• Employed a self-supervised, spatio-temporal backpropagation training method to leverage event camera characteristics.
• Validated the network's performance against existing methods on a public dataset.

Main Results:

• The proposed SNN achieves optical flow prediction accuracy comparable to the best existing methods.
• The SNN demonstrates a significant power saving of over 99% compared to current algorithms.
• The results highlight the feasibility of low-power hardware implementation for event camera optical flow.

Conclusions:

• The developed spike neural network effectively predicts optical flow for event cameras with high accuracy.
• This approach offers substantial power savings, making it ideal for resource-constrained hardware applications.
• The study paves the way for practical, low-power neuromorphic systems for real-time motion analysis in extreme environments.