Optimal sparsity in autoencoder memory models of the hippocampus.

Area of Science:

• Computational Neuroscience
• Neural Coding
• Memory Systems

Background:

• Efficient memory storage relies on compressed representations.
• Sparse autoencoders model neural compression, with activity mirroring hippocampal neurons.
• Previous models assumed fixed sparsity and uniform input compressibility.

Purpose of the Study:

• Investigate if input compressibility dictates optimal sparsity in sparse autoencoders.
• Explore the relationship between environmental statistics and neural sparsity.
• Provide theoretical grounding for dynamic sparsity tuning in the brain.

Main Methods:

• Simulated sparse autoencoders with varying input compressibility.
• Analyzed the impact of memory demand and compressibility on coding levels.
• Characterized the joint control of sparsity and enforcement strength on performance.

Main Results:

• Optimal coding level decreases with increased input compressibility.
• Divergence between desired and observed coding levels based on memory demand and compressibility.
• Weakly enforced sparsity maximizes optimal memory capacity.

Conclusions:

• Input compressibility is a key determinant of optimal sparsity in neural coding.
• Dynamic adjustment of sparsity in the hippocampus is predicted based on environmental statistics.
• Findings offer theoretical support for adaptive sparsity mechanisms in memory formation.