Sparse-Coding Variational Autoencoders.

Area of Science:

• Computational Neuroscience
• Computer Vision
• Machine Learning

Background:

• Sparse coding models efficiently represent visual stimuli using overcomplete dictionaries.
• Traditional sparse coding models face limitations in neural response computation and fitting due to recurrent dynamics and approximate inference.

Purpose of the Study:

• To introduce a novel Sparse Coding Variational Autoencoder (SVAE) framework.
• To address limitations of previous sparse coding models by incorporating a deep neural network-based recognition model.

Main Methods:

• Developed the SVAE by augmenting sparse coding with a probabilistic recognition model.
• Utilized a deep neural network for feedforward mapping from image patches to neural activities.
• Employed variational inference and maximization of the evidence lower bound (ELBO) for model fitting.

Main Results:

• The SVAE demonstrated superior test performance on natural image data compared to previous fitting methods.
• The recognition network captured nonlinear response properties consistent with early visual pathway neurons.
• Key differences from standard VAEs include an overcomplete latent representation, sparse/heavy-tailed priors, and a linear decoder.

Conclusions:

• The SVAE offers a neurally plausible and computationally efficient approach to sparse coding.
• This framework provides a principled method for fitting sparse coding models to data.
• The SVAE advances our understanding of efficient visual information processing in neural systems.