On the choice of a sparse prior

Area of Science:

• Computational neuroscience
• Visual system modeling
• Information theory

Background:

• Neurons are hypothesized to form sparse representations of natural stimuli for energy efficiency.
• Sparse coding in neural networks aims to minimize spikes, reducing energy consumption.
• Optimizing receptive fields for sparse responses is algorithmically equivalent to finding basis functions for sparse coding.

Purpose of the Study:

• To investigate the influence of sparse prior choices and constraints on neuronal receptive fields.
• To examine whether the specific sparse prior significantly affects estimated basis functions.
• To analyze the impact of unit variance normalization on neuronal activity and receptive field properties.

Main Methods:

• Simulated neurons with optimized receptive fields for sparse responses.
• Analysis of learning on a natural visual stimuli database (cat-cam).
• Inclusion of unit variance constraint into objective functions for analysis.

Main Results:

• Effective objective functions are dominated by the unit variance constraint, leading to similarity.
• Resulting receptive fields show both similarities and qualitative differences compared to simple cells.
• Distributions of coefficients are similar across different objective functions but do not match generative model priors.

Conclusions:

• The choice of sparse prior is relevant in modeling neuronal representations.
• Additional constraints, such as variance normalization, significantly influence receptive field properties.
• Sparse coding models need careful consideration of both priors and constraints for accurate biological representation.