Beyond ℓ1 sparse coding in V1.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Computer Vision

Background:

• Biological vision utilizes sparse neural activation for encoding stimuli.
• Generative models traditionally use the convex ℓ1 norm for biological sparsity approximation.
• The ℓ1 norm's convexity enables fast algorithmic solutions but may be suboptimal.

Purpose of the Study:

• To evaluate the performance of ℓ1 norm regularization against ℓp norms (0 ≤ p < 1) in modeling visual stimulus encoding.
• To compare the efficiency and reconstruction accuracy of ℓ0 and ℓ1 regularization in sparse coding models.
• To determine the optimal regularization strategy for efficient neural computation in the visual cortex.

Main Methods:

• Utilized biological vision as a test-bed for generative models.
• Compared performance of ℓ1 norm penalty with continuous relaxations of ℓp norms (0 ≤ p < 1).
• Employed a non-convex continuous relaxation of the ℓ0 pseudo-norm and compared it to ℓ1 regularization.

Main Results:

• The ℓ1 norm requires a dictionary ten times larger than the ℓ0 approach for equivalent stimulus reconstruction.
• Both ℓ0 and ℓ1 regularization produce receptive field shapes similar to biological V1 neurons.
• ℓ0-based regularization achieved approximately five times better stimulus reconstruction compared to ℓ1.

Conclusions:

• The soft thresholding of ℓ1 norm is suboptimal for sparse coding compared to ℓ0 pseudo-norm approximations.
• Efficient operation of the primary visual cortex (V1) likely employs regularization closer to ℓ0.
• Suggests a similar, potentially ℓ0-based, coding regime for the broader sensory cortex.