Deep time-delay reservoir computing: Dynamics and memory capacity

The deep time-delay reservoir computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems' distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to an adjustable distribution between the linear and nonlinear MC. Furthermore, numerical simulations show resonances between the clock cycle and delays of the layers in all degrees of MC. Contrary to MC losses in single-layer reservoirs, these resonances can boost separate degrees of MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.