Scaling Spike-Driven Transformer With Efficient Spike Firing Approximation Training.

Area of Science:

• Artificial Intelligence
• Neuromorphic Computing
• Computer Vision

Background:

• Spiking Neural Networks (SNNs) offer a low-power alternative to Artificial Neural Networks (ANNs).
• Challenges remain in bridging the performance gap and reducing training costs for SNNs.
• Binary firing mechanisms in spiking neurons can lead to performance limitations.

Purpose of the Study:

• To address the performance gap and high training costs of SNNs.
• To optimize spiking neuron firing patterns for improved efficiency and performance.
• To enable SNNs to serve as general-purpose visual backbones.

Main Methods:

• Proposed a Spike Firing Approximation (SFA) method using integer training and spike-driven inference.
• Developed an efficient spike-driven Transformer architecture.
• Introduced a spike-masked autoencoder to mitigate performance degradation during SNN scaling.

Main Results:

• Achieved state-of-the-art top-1 accuracy on ImageNet-1k, reaching 86.2% with a 173M parameter model.
• The 10M parameter SNN model outperformed existing SNNs by 7.2% on ImageNet.
• Demonstrated 4.5x training time acceleration and 3.9x inference energy efficiency improvements.
• Validated effectiveness across object detection, semantic segmentation, and neuromorphic vision tasks.

Conclusions:

• The SFA method enables SNNs to achieve competitive performance with ANNs.
• SNNs can now maintain their low-power advantage while matching ANN performance.
• This work represents a significant advancement towards SNNs as general visual backbones.