Noise Helps Optimization Escape From Saddle Points in the Synaptic Plasticity.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Experimental studies confirm inherent noise in the human brain.
• The functional role of this neural noise remains largely unknown.
• Stochasticity in neural systems poses computational challenges, potentially impairing brain function.

Purpose of the Study:

• To investigate the functional importance of noise in neural computation.
• To propose a biologically plausible noise structure for improving optimization in spiking neural networks.
• To demonstrate how noise can enhance learning and escape saddle points in high-dimensional optimization problems.

Main Methods:

• Proposed a biologically plausible noise structure for spiking neural networks.
• Utilized stochastic gradient descent for optimization.
• Deduced strict saddle conditions for synaptic plasticity.
• Provided biological interpretations based on the free energy principle and in vivo experiments.

Main Results:

• Noise significantly improves optimization performance in spiking neural networks.
• Noise aids in escaping saddle points in high-dimensional optimization domains under strict saddle conditions.
• Synaptic sampling with noise achieved nearly 20% higher accuracy on synthetic datasets.
• Accuracy gains of at least 10% were observed on MNIST and CIFAR-10 datasets with noise.

Conclusions:

• Noise plays a crucial role in efficient brain computation and learning.
• The proposed noise structure offers a new learning framework for the brain.
• This research provides insights into deep noisy spiking neural networks and their potential applications.