Spike-Driven Neighborhood Attention for Vision Transformers

Spiking Neural Networks (SNNs) promise ultra-low-power, event-driven computation; however, current efforts to integrate self-attention into spiking transformers predominantly rely on global attention mechanisms, introducing growing token complexity when scaling and diluting discriminative feature representations. We introduce Spike-Driven Neighborhood Attention (SDNA) over the full k×k neighborhood in two forms: Pairwise SDNA, which gates each query-neighbor interaction before aggregating, and Linear SDNA, which exploits the absence of a softmax in spiking attention to pool the key-value interaction over the window and gate it once, evaluating the spiking gate per position rather than per query-neighbor pair. Both operators are placed in a fully local hierarchical backbone that increases the receptive field without any global attention layer. Across static and neuromorphic benchmarks the two operators are competitive with global spike-driven transformers at comparable or lower parameter counts. Pairwise SDNA achieves the highest accuracy among the compared spike-driven transformers on CIFAR-10 at 97.24%, while Linear SDNA achieves the highest accuracy on N-Caltech101 at 89.65%. These results show that competitive performance can be achieved using a fully local hierarchical attention design, without requiring an explicit global-attention stage.