Dimension of Activity in Random Neural Networks

Area of Science:

• Computational Neuroscience
• Machine Learning Theory
• Complex Systems

Background:

• Neural networks are complex dynamical systems crucial for information processing.
• Understanding coordinated activity, like cross-covariances, is key to deciphering network function and learning.
• Dynamical Mean Field Theory (DMFT) has explained chaotic activity but lacked cross-covariance calculations.

Purpose of the Study:

• To develop and apply a self-consistent dynamical mean field theory (DMFT) for calculating cross-covariances in random neural networks.
• To investigate spatiotemporal activity coordination in a canonical random network model.
• To generalize the theoretical framework to non-independent and identically distributed (non-i.i.d.) couplings.

Main Methods:

• Employed a two-site cavity dynamical mean field theory (DMFT) approach for self-consistent cross-covariance calculations.
• Analyzed a classic random neural network model with independent and identically distributed (i.i.d.) couplings.
• Extended the analysis to networks with partially symmetric, non-i.i.d. couplings.

Main Results:

• Demonstrated that neural activity exhibits an extensive but fractionally low effective dimension.
• Identified a long population-level timescale governing activity dynamics.
• Provided a generalized theoretical framework applicable to diverse single-unit dynamics and coupling structures.

Conclusions:

• The developed DMFT method successfully quantifies cross-covariances, offering new analytical tools for neural network dynamics.
• The findings illuminate fundamental properties of coordinated activity in random networks, relevant to both biological and artificial systems.
• The theoretical framework's flexibility allows for the study of more biologically and computationally realistic network structures.