Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking Neural Networks (SNNs) offer a promising computing paradigm but face challenges in training deep architectures.
• Existing methods like Artificial Neural Network (ANN)-to-SNN conversion do not fully capture temporal dynamics.
• Directly training deep SNNs with spike events is difficult due to non-differentiable spike generation.

Purpose of the Study:

• To develop a method for directly training deep Spiking Neural Networks (SNNs) using input spike events.
• To enable effective spike-based backpropagation for deep SNNs.
• To demonstrate the efficacy of the proposed method for both training and inference.

Main Methods:

• Proposed an approximate derivative method accounting for the leaky behavior of Leaky Integrate-and-Fire (LIF) neurons.
• Enabled spike-based backpropagation for training deep convolutional SNNs.
• Evaluated the method on VGG and Residual architectures using MNIST, SVHN, and CIFAR-10 datasets.

Main Results:

• Achieved state-of-the-art classification accuracies on MNIST, SVHN, and CIFAR-10 datasets compared to other spike-based trained SNNs.
• Demonstrated the effectiveness of the proposed spike-based learning on deep network architectures.
• Analyzed sparse event-based computations to show efficacy for inference.

Conclusions:

• The proposed approximate derivative method facilitates direct training of deep SNNs with spike events.
• This approach overcomes limitations of previous SNN training methods, capturing temporal dynamics effectively.
• The method shows significant promise for efficient, event-based inference in SNNs.