A regularization perspective based theoretical analysis for adversarial robustness of deep spiking neural networks

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking Neural Networks (SNNs) are the third generation of neural networks.
• Converting pre-trained Artificial Neural Networks (ANNs) to SNNs is computationally efficient but results in vulnerability to adversarial attacks.
• While direct training improves SNN robustness, the underlying theoretical mechanisms remain unclear.

Purpose of the Study:

• To provide a theoretical explanation for the adversarial robustness of directly trained SNNs.
• To analyze the expected risk function and identify mechanisms contributing to robustness.
• To demonstrate the effectiveness of this approach through empirical validation.

Main Methods:

• Modeling the stochastic process of Poisson encoders in SNNs.
• Theoretical analysis of the expected risk function to identify a positive semidefinite regularizer.
• Conducting extensive experiments on CIFAR10 and CIFAR100 datasets to validate theoretical findings.

Main Results:

• A theoretical proof reveals a positive semidefinite regularizer in trained SNNs.
• This regularizer effectively reduces the gradients of the output with respect to the input, enhancing robustness.
• Experimental results show significantly smaller gradient magnitudes in trained SNNs compared to converted SNNs (13-160 times less).

Conclusions:

• Directly training SNNs introduces an inherent robustness mechanism through a specific regularizer.
• Minimizing the sum of squared gradients is key to improving adversarial resilience in SNNs.
• The findings offer a theoretical foundation for developing more robust SNNs against adversarial attacks.