DelGrad: exact event-based gradients for training delays and weights on spiking neuromorphic hardware

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Neuromorphic Engineering

Background:

• Spiking Neural Networks (SNNs) use signal timing for information processing.
• Trainable transmission delays in SNNs improve accuracy and efficiency.
• Current training methods for SNNs with delays are imprecise and resource-intensive.

Purpose of the Study:

• To develop an analytical, event-based training method for SNNs with trainable delays.
• To compute exact loss gradients for synaptic weights and transmission delays.
• To enable efficient SNN training on neuromorphic hardware.

Main Methods:

• Proposed DelGrad, an analytical, event-based training algorithm.
• Computed exact loss gradients based purely on spike timing.
• Implemented DelGrad on the BrainScaleS-2 neuromorphic platform.

Main Results:

• Demonstrated DelGrad's ability to optimize both weights and delays.
• Showcased parameter efficiency and accuracy benefits of SNNs with delays.
• Experimentally validated the stabilizing effect of delays on noisy hardware for the first time.

Conclusions:

• DelGrad offers a precise and efficient method for training SNNs with delays.
• The method eliminates the need for internal variable tracking, suiting neuromorphic hardware.
• DelGrad significantly advances SNN training for neuromorphic substrates.