Dynamically Optimizing Network Structure Based on Synaptic Pruning in the Brain.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Neural network architecture design is often empirical, risking overfitting or underfitting.
• Fully connected networks incur high computational and storage costs, hindering mobile deployment.
• Existing pruning methods often reduce accuracy when optimizing deep convolutional networks.

Purpose of the Study:

• To propose a brain-inspired synaptic pruning method for dynamic neural network architecture optimization.
• To improve network performance and reduce computational overhead.
• To enable efficient deployment of neural networks on resource-constrained devices.

Main Methods:

• A biologically inspired synaptic pruning technique dynamically eliminates redundant connections.
• Connections are pruned based on consecutive low or no activation, mimicking brain development.
• The method is tested on classification tasks using MNIST, Fashion MNIST, and CIFAR-10 datasets.

Main Results:

• The proposed method effectively prunes connections, removing 59-90% even in compact networks.
• Experimental results show improvements in learning speed and classification accuracy.
• The approach demonstrates effectiveness across various classification task complexities.

Conclusions:

• Brain-inspired synaptic pruning offers an effective way to optimize neural network architectures dynamically.
• This method enhances network performance and efficiency, addressing limitations of traditional approaches.
• The technique holds promise for deploying sophisticated neural networks on mobile and edge devices.