Deep time-delay reservoir computing: Dynamics and memory capacity

Area of Science:

• Computational neuroscience
• Machine learning
• Nonlinear dynamics

Background:

• Reservoir computing (RC) is a machine learning paradigm utilizing recurrent neural networks with fixed weights.
• Time-delay reservoir computing (TDRC) incorporates time delays into the recurrent connections, enhancing computational capabilities.
• Deep TDRC architectures offer increased complexity and potential for sophisticated information processing.

Purpose of the Study:

• To investigate the relationship between the dynamical properties of a deep Ikeda-based reservoir and its memory capacity (MC).
• To explore how these dynamical properties can be leveraged for optimizing TDRC performance.
• To identify configurations that maximize specific degrees of MC for targeted applications.

Main Methods:

• Analysis of bifurcations in the autonomous system underlying the deep Ikeda reservoir.
• Computation of conditional Lyapunov exponents to quantify generalized synchronization between input and layer dynamics.
• Numerical simulations to observe the impact of resonances between clock cycles and layer delays on MC.

Main Results:

• Memory capacity (MC) is directly related to the system's proximity to bifurcations and the magnitude of conditional Lyapunov exponents.
• The interplay of different dynamical regimes allows for adjustable distributions between linear and nonlinear MC.
• Resonances between clock cycle and delays can enhance, rather than degrade, MC in deep TDRC, unlike in single-layer systems.

Conclusions:

• The dynamical properties of deep TDRC systems provide a mechanism for optimizing memory capacity.
• Specific configurations can be designed to achieve either high nonlinear MC or extended linear MC.
• Understanding these dynamics enables the creation of specialized TDRC systems for diverse computational tasks.