Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neuromorphic Computing

Background:

• Spiking Neural Networks (SNNs) offer energy-efficient processing on neuromorphic hardware due to their sparse, event-driven nature.
• However, SNNs face training instability caused by variations in forward activations and backward gradients over time.

Purpose of the Study:

• To enhance the training stability and performance of Spiking Neural Networks.
• To introduce a novel temporal Batch Normalization technique tailored for SNNs.

Main Methods:

• Propose Batch Normalization Through Time (BNTT), a temporal adaptation of Batch Normalization for SNNs.
• Decouple BN parameters along the time axis to better capture temporal spike dynamics.
• Evaluate BNTT on diverse datasets including CIFAR-10, CIFAR-100, Tiny-ImageNet, DVS-CIFAR10, and Sequential MNIST.

Main Results:

• BNTT achieves near state-of-the-art performance across multiple benchmark datasets.
• Demonstrate improved robustness against random and adversarial noise.
• Identify temporal early exit as a capability, reducing inference latency by 5-20 time-steps.

Conclusions:

• BNTT effectively addresses SNN training instability by adapting normalization parameters over time.
• The technique enhances model robustness and enables significant reductions in inference latency.
• BNTT represents a promising advancement for practical SNN deployment.