Towards parameter-free attentional spiking neural networks

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking neural networks (SNNs) offer energy efficiency and spatiotemporal modeling for neuromorphic hardware.
• Attentional modules improve SNNs for sequential data but increase memory consumption.
• High memory usage of parameterized attention hinders SNNs on resource-constrained neuromorphic chips.

Purpose of the Study:

• To develop a parameter-free attention (PfA) mechanism for SNNs.
• To enhance SNN feature representation and performance without additional memory overhead.
• To improve SNNs' suitability for energy-efficient neuromorphic computing.

Main Methods:

• Introduced a parameter-free attention (PfA) mechanism.
• Integrated PfA into spiking neurons to bolster feature representation.
• Evaluated PfA-SNNs on diverse datasets including SHD, BAE-TIDIGITS, SSC, DVS-Gesture, DVS-Cifar10, Cifar10, and Cifar100.

Main Results:

• PfA-SNNs achieved competitive or superior classification accuracy across multiple datasets.
• The proposed PfA mechanism enhanced performance without increasing model parameters.
• PfA-SNNs demonstrated improved noise robustness compared to conventional SNNs and those with parameterized attention.

Conclusions:

• The parameter-free attention (PfA) mechanism effectively enhances SNN performance and efficiency.
• PfA is a viable solution for memory-constrained neuromorphic hardware.
• PfA-SNNs represent a significant advancement in energy-efficient AI for spatiotemporal data.