Efficient spiking neural network design via neural architecture search

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking neural networks (SNNs) offer energy efficiency through brain-inspired sparse, event-driven communication.
• Manual design of SNN architectures is labor-intensive and prone to errors, hindering performance.
• Automated machine learning, specifically neural architecture search (NAS), has advanced deep learning but faces challenges with SNNs.

Purpose of the Study:

• To develop an automated spiking neural architecture search (NAS) method for efficient SNNs.
• To address the prolonged search times typically associated with NAS for SNNs.
• To optimize SNNs for both high accuracy and computational efficiency within budget constraints.

Main Methods:

• Proposed a novel NAS method encoding candidate SNN architectures within a branchless spiking supernet to reduce computational load.
• Introduced Synaptic Operation (SynOps)-aware optimization to identify computationally efficient subspaces within the supernet.
• Validated the method through experiments assessing search efficiency, accuracy, and computational cost.

Main Results:

• The proposed NAS method significantly reduced search time compared to existing approaches.
• Discovered SNNs demonstrated superior accuracy and lower computational cost than state-of-the-art SNNs.
• Experimental validation confirmed the effectiveness of the search process and the accuracy-computational cost trade-off.

Conclusions:

• Automated NAS can effectively discover efficient SNN architectures, overcoming manual design limitations.
• The branchless supernet and SynOps-aware optimization are key to efficient SNN architecture discovery.
• This approach enables the development of high-performance, energy-efficient SNNs for real-world applications.