BlocTrain: Block-Wise Conditional Training and Inference for Efficient Spike-Based Deep Learning.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Deep Learning

Background:

• Spiking neural networks (SNNs) show promise for intelligent autonomous systems due to their efficient, sparse, spike-based processing.
• End-to-end training of deep SNNs is computationally expensive and memory-intensive due to backpropagation through time.
• Existing training methods struggle with the scale and complexity of deep SNNs.

Purpose of the Study:

• To introduce BlocTrain, a novel algorithm for scalable and memory-efficient training of deep SNNs.
• To address the computational and memory challenges associated with training deep SNNs.
• To enable more efficient development of SNNs for advanced AI applications.

Main Methods:

• BlocTrain segments deep SNNs into blocks, training them sequentially using local errors.
• The algorithm identifies easy versus hard classes post-block training to focus subsequent training on difficult classes.
• A hard class detector (HCD) is integrated for early inference exits on easy classes, activating deeper blocks only for hard classes.

Main Results:

• BlocTrain achieved 86.4% accuracy on CIFAR-10 using a ResNet-9 SNN, with up to 2.95x lower memory usage during training.
• Inference showed 1.89x compute efficiency due to the early exit strategy, with a 1.45x memory overhead.
• A ResNet-11 SNN trained on CIFAR-100 achieved 58.21% accuracy, notable for spike-based backpropagation.

Conclusions:

• BlocTrain provides a scalable and memory-efficient method for training deep SNNs.
• The block-wise training and early exit strategy significantly improve training efficiency and inference speed.
• This approach facilitates the development of more complex SNNs for real-world AI tasks.