Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Synaptic unreliability is a key factor in cortical stochasticity.
• Existing neural network models often lack efficient unsupervised learning mechanisms.

Purpose of the Study:

• Introduce Synaptic Sampling Machines (S2Ms) as a novel neural network model.
• Explore the use of synaptic stochasticity for Monte Carlo sampling and unsupervised learning.
• Evaluate S2Ms' performance and robustness in benchmark classification tasks.

Main Methods:

• Developed S2Ms, a class of neural network models utilizing synaptic stochasticity.
• Employed a local synaptic plasticity rule with event-driven contrastive divergence for online learning.
• Tested S2Ms with both discrete-timed artificial units and continuous-timed leaky integrate-and-fire neurons.

Main Results:

• S2Ms effectively perform Monte Carlo sampling and unsupervised learning.
• Learned representations are sparse, robust to reduced bit precision, and synapse pruning (>75% connection removal).
• Spiking neuron-based S2Ms outperform existing spike-based unsupervised learners.

Conclusions:

• S2Ms offer an efficient and robust approach to unsupervised learning and generative modeling.
• The models show promise for on-line learning in power-efficient, brain-inspired hardware.
• Synaptic stochasticity serves as a dual-purpose mechanism for sampling and regularization.