Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization.

Area of Science:

• Artificial Intelligence
• Computer Science
• Computational Neuroscience

Background:

• Spiking neural networks (SNNs) and artificial neural networks (ANNs) face significant memory and compute challenges, limiting their deployment on edge devices.
• Model compression techniques, primarily used for ANNs, offer a path to improve efficiency by reducing parameters and operations.
• SNN compression is complex due to accuracy sensitivity and the event-driven nature of spike computation, presenting unique challenges.

Purpose of the Study:

• To develop a comprehensive compression methodology for SNNs that minimizes accuracy loss.
• To address the challenges of SNN compression, including accuracy sensitivity and the dynamic nature of spike events.
• To enable efficient deployment of SNNs on resource-constrained edge devices.

Main Methods:

• Formulated connection pruning and weight quantization as a constrained optimization problem.
• Combined spatiotemporal backpropagation (STBP) with the alternating direction method of multipliers (ADMMs) for optimization.
• Introduced activity regularization to reduce spike events and computational load.

Main Results:

• Achieved significant SNN compression through single or joint application of pruning, quantization, and regularization techniques.
• Demonstrated minimal accuracy loss across various pattern recognition tasks (MNIST, N-MNIST, CIFAR10, CIFAR100).
• Developed quantitative metrics to effectively evaluate SNN compression performance.

Conclusions:

• The proposed comprehensive SNN compression methodology effectively reduces memory and compute costs.
• This approach enables efficient SNN deployment on edge devices by addressing accuracy and computational challenges.
• This work represents a novel and comprehensive study on SNN compression, achieving superior results by optimizing all compressible components.