Related Experiment Videos
Convergence of stochastic learning in perceptrons with binary synapses.
1Department of Physiology, University of Bern, CH-3012 Bern, Switzerland. wsenn@cns.unibe.ch
Summary
This study shows that slow stochastic learning with binary synapses can enable high-probability memory storage and pattern classification. This approach overcomes limitations of discrete synaptic states in neural networks, even for complex patterns.
Area of Science:
- Computational Neuroscience
- Machine Learning
- Artificial Intelligence
Background:
- Biological synapses have discrete states, limiting neural network storage capacity during online learning.
- The 'palimpsest property' describes how old memories decay exponentially with new ones.
- Finding optimal discrete synaptic strengths for pattern classification is computationally challenging (NP-complete).
Purpose of the Study:
- To demonstrate the feasibility of learning with discrete (binary) synapses.
- To introduce and analyze a 'slow stochastic learning' algorithm.
- To show convergence for linearly separable patterns and application to nonlinearly separable ones.
Main Methods:
- Modifying a random fraction of synapses after each stimulus presentation.
- Implementing a constraint where synapses change only if the output neuron errs (similar to perceptron learning).
- Proving convergence for linearly separable patterns with sufficient neurons.
Main Results:
- Slow stochastic learning enables high-probability learning with binary synapses.
- The algorithm converges reliably for linearly separable patterns.
- Successful application to nonlinearly separable patterns using multiple output units, achieving comparable performance to optimal methods.
Conclusions:
- Discrete synaptic states do not inherently prevent effective learning and memory storage.
- Slow stochastic learning offers a viable approach for neural networks with binary synapses.
- This method provides a robust solution for pattern classification tasks, including complex ones.