On unique representations of certain dynamical systems produced by continuous-time recurrent neural networks

Area of Science:

• Computational neuroscience
• Machine learning theory
• Dynamical systems theory

Background:

• Feedforward neural networks (FNNs) have known redundancies in function description.
• Dynamical systems (DSs) are crucial in modeling complex phenomena.
• Recurrent neural networks (RNNs) are powerful tools for approximating DSs.

Purpose of the Study:

• To extend the understanding of redundancy from FNNs to RNNs for DS description.
• To analyze the redundancy inherent in describing affine neural dynamical systems (A-NDSs) using RNNs and affine maps.
• To determine the uniqueness of RNN-affine map pairs for a given A-NDS.

Main Methods:

• Mathematical analysis of RNNs and affine maps.
• Development of the concept of an n-dimensional affine neural dynamical system (A-NDS).
• Investigation of the relationship between A-NDSs and the RNNs that generate them.

Main Results:

• Clarification of the extent to which an RNN and an affine map are uniquely determined by their corresponding A-NDS.
• Identification of a non-redundant sufficient search set for DS approximation problems.
• Characterization of redundancy in RNN-based DS approximation.

Conclusions:

• The study provides a theoretical framework for understanding redundancy in RNNs for DS approximation.
• Results contribute to more efficient and accurate modeling of dynamical systems.
• Findings offer insights into the unique representation capabilities of RNNs.