Slow Transition to Low-Dimensional Chaos in Heavy-Tailed Recurrent Neural Networks.

Area of Science:

• Computational neuroscience
• Theoretical neuroscience
• Machine learning

Background:

• Synaptic weights in the brain often follow heavy-tailed distributions.
• Theoretical analyses of recurrent neural networks (RNNs) typically assume Gaussian connectivity, which may not be biologically accurate.

Purpose of the Study:

• To investigate the dynamics of RNNs with synaptic weights drawn from heavy-tailed Lévy alpha-stable distributions.
• To analyze the transition between quiescent and chaotic states in finite-size heavy-tailed RNNs.
• To understand the tradeoff between robustness and dimensionality in biologically realistic neural networks.

Main Methods:

• Systematic study of RNN activity with Lévy alpha-stable distributed random weights.
• Finite-size analysis to identify the transition point from quiescent to chaotic dynamics.
• Theoretical prediction and simulation-based validation of the transition gain.
• Analysis of the Lyapunov dimension of attractors to assess effective dimensionality.

Main Results:

• Finite heavy-tailed RNNs exhibit a sharp transition between quiescent and chaotic dynamics, contrasting with mean-field predictions of ubiquitous chaos.
• A broader gain regime near the edge of chaos is observed in heavy-tailed RNNs, indicating a slower transition to chaos.
• Heavier tails in synaptic weights lead to a reduced Lyapunov dimension, implying lower effective dimensionality of the neural activity.

Conclusions:

• Heavy-tailed connectivity in RNNs provides a biologically plausible model with a robust transition to chaos.
• A tradeoff exists between the robustness of dynamics near the edge of chaos and the richness (dimensionality) of neural activity.
• The study offers a tractable framework for analyzing dynamics in realistically sized, heavy-tailed neural circuits by characterizing the transition point in finite-size networks.