Entropy, Inhibition and Memory in Balanced Spiking Reservoirs

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Recurrent neural networks (RNNs) are explored as both task-performing machine learning models and computationally realistic models of cortical circuits.
• Reservoir computing provides a potential bridge between these approaches, but the relationship between dynamical regimes and computational performance in biologically constrained spiking networks remains unclear.

Purpose of the Study:

• To systematically map the link between dynamical regimes and computational performance in biologically constrained spiking RNNs.
• To characterize separation capacity and transient memory across the phase diagram of a balanced excitatory-inhibitory network.

Main Methods:

• The Brunel balanced excitatory-inhibitory network was modeled as a reservoir.
• Separation capacity (kernel quality) and transient memory (corrected linear memory capacity) were evaluated.
• A four-state Markov source with fixed marginal entropy was used to set the Shannon entropy rate.

Main Results:

• Both separation capacity and transient memory increased monotonically with the inhibitory ratio (g).
• Optimal performance was observed in the asynchronous irregular regime, with diminishing increments.
• Sparse input coupling was crucial for memory; dense coupling erased memory across all regimes.

Conclusions:

• Inhibitory balance serves as a unified architectural control parameter for RNNs.
• This finding offers a quantitative design criterion for both computational neuroscience models and reservoir computing applications.
• The asynchronous irregular regime, controlled by inhibitory balance, is critical for robust neural computation.