Multiscale analysis of slow-fast neuronal learning models with noise

Area of Science:

• Computational neuroscience
• Machine learning theory
• Mathematical physics

Background:

• Recurrent neural networks (RNNs) exhibit complex dynamics involving neuronal activity and synaptic plasticity.
• Understanding unsupervised learning in RNNs requires analyzing multiple timescales: fast neuronal dynamics, intermediate input signals, and slow learning processes.

Purpose of the Study:

• To apply temporal averaging methods to analyze RNNs with unsupervised learning.
• To derive a reduced deterministic model for connectivity dynamics by exploiting timescale separation.
• To investigate the relationship between learning rules and the resulting network connectivity.

Main Methods:

• Application of stochastic averaging theory with periodic forcing.
• Derivation of a reduced deterministic model for connectivity.
• Analysis of linear activity models and various learning rules (Hebbian, trace, anti-symmetric).
• Asymptotic approximation of equilibrium connectivity in a weakly connected regime.

Main Results:

• The symmetric part of the learned connectivity matrix encodes input correlation structure.
• The anti-symmetric part encodes cross-correlations between inputs and their time derivatives.
• The ratio of timescales is a critical parameter for understanding temporal correlations captured by the network.

Conclusions:

• Temporal averaging effectively simplifies complex RNN models with learning.
• The derived reduced model provides insights into how RNNs store information about input statistics and temporal dependencies.
• The study highlights the importance of timescale separation in analyzing learning in neural networks.