Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Spiking neural networks (SNNs) encode spatio-temporal information, mimicking brain-like behaviors.
• Current SNN training methods often prioritize spatial over temporal dynamics, leading to performance bottlenecks.
• The non-differentiable nature of spike activity complicates supervised training in SNNs.

Purpose of the Study:

• To introduce a novel spat-temporal backpropagation (STBP) algorithm for high-performance SNN training.
• To address the non-differentiability challenge in SNNs using an approximated derivative suitable for gradient descent.
• To enhance SNNs' ability to process rich spatio-temporal dynamics for brain-like computing.

Main Methods:

• Developed a spat-temporal backpropagation (STBP) algorithm combining spatial domain (SD) and temporal domain (TD) information.
• Proposed an approximated derivative for spike activity to enable gradient descent training.
• Evaluated STBP on fully connected and convolutional architectures using static (MNIST) and dynamic (N-MNIST) datasets, including object detection.

Main Results:

• The STBP algorithm achieved superior accuracy compared to existing state-of-the-art methods on spiking networks.
• Demonstrated effective training of SNNs by integrating both spatial and temporal information without complex additional techniques.
• Validated the approach across diverse datasets, highlighting its robustness for various SNN architectures.

Conclusions:

• The STBP algorithm offers a new perspective for training high-performance SNNs.
• This method effectively captures spatio-temporal dynamics crucial for advanced brain-like computing.
• STBP provides a simplified yet powerful approach to overcome SNN training limitations.