Enhancing spiking transformers with temporal feedback coding and global-local dynamic neurons

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Computer Vision

Background:

• Spiking Neural Networks (SNNs) offer energy efficiency and biological plausibility over Artificial Neural Networks (ANNs).
• Transformer architectures integrated with SNNs show promise in vision tasks.
• Existing Transformer-based SNNs face challenges like spike pattern collapse and limited temporal modeling due to direct coding and standard Leaky Integrate-and-Fire (LIF) neurons.

Purpose of the Study:

• To address limitations in current Transformer-based SNNs, specifically spike pattern collapse and insufficient temporal modeling.
• To propose novel components that enhance the performance and temporal capacity of SNNs.

Main Methods:

• Introduced a Temporal Feedback Coding (TFC) scheme to diversify spike patterns via feedback during encoding.
• Developed a Global-Local Dynamic LIF (GLD-LIF) neuron to improve cross-step dependency modeling.
• Conducted extensive experiments on three Transformer-based SNN backbones across five datasets and various time steps.

Main Results:

• Achieved consistent performance improvements across different Transformer-based SNN architectures and datasets.
• Reported accuracy gains of up to 3.64% on N-Caltech101 and 1.02% on ImageNet-1K.
• Demonstrated effectiveness through analyses of spike patterns, attention, temporal robustness, and corruption resilience.

Conclusions:

• The proposed TFC scheme and GLD-LIF neuron effectively enhance Transformer-based SNNs.
• The approach improves temporal modeling capacity and accuracy while maintaining broad compatibility.
• These advancements contribute to more capable and efficient SNNs for vision applications.