Optimizing event-driven spiking neural network with regularization and cutoff.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking neural networks (SNNs) mimic natural neural networks for enhanced computational efficiency.
• Current SNNs infer over fixed durations, limiting dynamic inference capabilities.
• Event-driven processing offers potential for more efficient neural network operations.

Purpose of the Study:

• To introduce a cutoff mechanism for SNNs to enable dynamic inference and improve computational efficiency.
• To propose novel optimization techniques for inference-efficient SNNs.
• To enhance the relationship between SNNs and event-driven processing.

Main Methods:

• Proposed a cutoff mechanism to terminate SNN inference dynamically.
• Introduced two optimization techniques: Top-K cutoff and regularization.
• Conducted experiments on diverse frame-based and event-based datasets (CIFAR10/100, Tiny-ImageNet, CIFAR10-DVS, N-Caltech101, DVS128 Gesture).

Main Results:

• Achieved 1.76 to 2.76x fewer timesteps on CIFAR-10.
• Reduced timesteps by 1.64 to 1.95x across event-based datasets.
• Maintained near-zero accuracy loss with the proposed techniques.

Conclusions:

• The proposed cutoff and regularization techniques effectively enhance SNN inference efficiency.
• These methods are compatible with both ANN-to-SNN conversion and direct training approaches.
• The findings highlight the potential of dynamic inference in SNNs for reduced latency and improved accuracy.