A network of spiking neurons for computing sparse representations in an energy-efficient way

Area of Science:

• Computational Neuroscience
• Applied Mathematics
• Signal Processing

Background:

• Sparse redundant representations are crucial in applied mathematics and neuroscience.
• Energy-efficient computation of these representations is a significant challenge.
• Existing algorithms may not meet the demands for low-power, distributed systems.

Purpose of the Study:

• To propose a novel hybrid distributed algorithm (HDA) for energy-efficient sparse representation computation.
• To investigate the algorithm's performance and stability in a distributed network setting.
• To explore the potential of HDA as a model for neural computation.

Main Methods:

• Developed a hybrid distributed algorithm (HDA) operating on a network of simple nodes.
• Nodes utilize low-bandwidth communication channels.
• HDA integrates gradient-descent-like steps on analog variables and coordinate-descent-like steps on quantized variables.

Main Results:

• HDA achieves numerical performance on par with existing state-of-the-art algorithms.
• Representation error decays as 1/t in the asymptotic regime.
• The algorithm exhibits stability against time-varying noise, with error decaying as 1/√t for Gaussian white noise.

Conclusions:

• HDA offers an energy-efficient solution for computing sparse redundant representations in distributed systems.
• The algorithm's structure is analogous to integrate-and-fire neural networks, suggesting potential as a neural computation model.
• HDA provides robust and scalable performance for sparse representation tasks.