Probing the Relationship Between Latent Linear Dynamical Systems and Low-Rank Recurrent Neural Network Models

Area of Science:

• Computational Neuroscience
• Machine Learning
• Dynamical Systems Theory

Background:

• Neural population activity during behavior is often characterized by low-dimensional dynamics.
• Latent linear dynamical system (LDS) models and low-rank recurrent neural networks (RNNs) are two common approaches for modeling these dynamics.
• These modeling approaches share similarities but are applied in different contexts.

Purpose of the Study:

• To precisely examine the relationship between latent LDS models and linear low-rank RNNs.
• To determine the conditions under which one model class can be converted into the other.

Main Methods:

• Mathematical analysis of the conversion between latent LDS models and linear low-rank RNNs.
• Investigation of the non-Markovian properties of latent LDS models in relation to RNNs.
• Mapping linear RNNs onto LDS models and comparing latent dimensionalities.

Main Results:

• Latent LDS models can only be converted to RNNs under specific limiting conditions due to non-Markovian properties.
• Linear RNNs can be mapped to LDS models, with the latent dimensionality being at most twice the RNN's rank.
• Partially observed RNNs are more effectively represented by LDS models than by RNNs using only observed units.

Conclusions:

• The conversion between latent LDS and linear RNN models is asymmetric.
• LDS models offer a potentially more suitable framework for representing partially observed RNN dynamics.
• Understanding these model relationships is crucial for accurate neural data analysis and interpretation.