A Learning Framework for Winner-Take-All Networks with Stochastic Synapses.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Recent generative models utilize neural networks to transform noise into complex data distributions.
• Biological neural networks may employ similar principles, transforming intrinsic noise into environmental models.
• Differences in biological noise processes and neural signaling (spikes, local interactions) pose challenges for current generative frameworks.

Purpose of the Study:

• To develop a biologically motivated generative model that overcomes limitations of current abstract generative networks.
• To investigate if multilayer winner-take-all circuits with stochastic synapses can learn data distributions.
• To extend the applicability of modern stochastic network architectures to biologically realistic scenarios.

Main Methods:

• Developed a biologically motivated model featuring multilayer winner-take-all circuits and stochastic synapses.
• Derived an approximate analytical description for the proposed network dynamics.
• Employed a variational learning setting with stochastic backpropagation to optimize data log-likelihood.

Main Results:

• The proposed biologically motivated model admits an approximate analytical description, enabling variational learning.
• The network successfully learned a generative model of the data through stochastic backpropagation.
• Demonstrated the model's versatility in structured output prediction and semi-supervised learning tasks.

Conclusions:

• Biologically plausible generative models can be trained using stochastic backpropagation, even with synaptic transmission failure as the primary noise mechanism.
• This work bridges the gap between abstract generative models and biologically realistic neural networks.
• The findings expand the scope of modern stochastic network architectures for neuroscience and machine learning applications.