Developmental Plasticity-Inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networks

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Neuroscience

Background:

• Deep artificial neural networks (ANNs) and spiking neural networks (SNNs) lack efficient compression methods inspired by brain developmental plasticity.
• Current methods limit the ability of ANNs and SNNs to learn efficiently, rapidly, and accurately in dynamic environments.

Purpose of the Study:

• To propose a developmental plasticity-inspired adaptive pruning (DPAP) method for optimizing network structures during learning.
• To enhance the efficiency, speed, and accuracy of ANNs and SNNs by incorporating biologically realistic mechanisms.

Main Methods:

• Developed the DPAP method inspired by the "use it or lose it, gradually decay" principle of dendritic spine, synapse, and neuron pruning.
• Integrated biologically realistic mechanisms: dendritic spine dynamic plasticity, activity-dependent neural spiking trace, and local synaptic plasticity.
• Implemented an adaptive pruning strategy for dynamic network optimization without pre-training or retraining.

Main Results:

• DPAP demonstrated consistent and remarkable performance and speed enhancements across diverse benchmark tasks for ANNs and SNNs.
• Achieved significant network compression, leading to improved efficiency.
• Showcased superior results with spatio-temporal joint pruning for SNNs in neuromorphic datasets, reaching state-of-the-art (SOTA) performance.

Conclusions:

• Developmental plasticity can enable complex deep networks to evolve into brain-like efficient and compact structures.
• The DPAP method offers a biologically plausible approach to optimize deep networks, achieving SOTA performance for SNNs.
• This research bridges the gap between neuroscience and artificial intelligence, paving the way for more efficient and adaptive AI systems.