Memory capacity of networks with stochastic binary synapses.

Area of Science:

• Computational neuroscience
• Theoretical neuroscience
• Machine learning

Background:

• Attractor neural networks store patterns in synaptic matrices, acting as fixed point attractors.
• Storage capacity is measured by the number of patterns and information per synapse.
• Understanding capacity limits is crucial for neural network and biological memory research.

Purpose of the Study:

• To compute storage capacity (number of patterns and bits/synapse) in fully connected binary neural networks.
• To analyze capacity in large and sparse coding limits with finite-size corrections.
• To compare different learning scenarios (Willshaw, one-shot, multi-shot stochastic learning).

Main Methods:

• Analytical computation of storage capacity in binary neural networks.
• Derivation of finite-size corrections for capacity.
• Application and comparison of models including Willshaw, one-shot, and multi-shot learning.

Main Results:

• Computed storage capacities in large and sparse coding limits for binary neural networks.
• Derived finite-size corrections that match simulation results for large networks.
• Demonstrated that finite-size effects substantially reduce storage capacity, even in realistic network sizes.

Conclusions:

• Finite-size effects are a critical factor limiting storage capacity in neural networks.
• The study provides a framework for comparing different neural network models.
• Results offer insights into memory storage mechanisms in biological systems like the hippocampus and cortex.