Transient times and cycle-rich topology in reservoir computing

Area of Science:

• Computational Neuroscience
• Machine Learning
• Complex Systems

Background:

• Reservoir computing (RC) training necessitates an initial synchronization period for the reservoir's internal state with the input driver.
• The duration of this transient, known as the washout period, is crucial: insufficient length leads to training failure, while excessive length wastes valuable data.
• Existing methods often treat washout as an empirical parameter, lacking a deep understanding of its underlying dynamics.

Purpose of the Study:

• To quantify synchronization transients in reservoir computing.
• To elucidate the factors influencing transient length, particularly the spectral radius and network topology.
• To develop practical design rules for optimizing reservoir computing performance and data efficiency.

Main Methods:

• Derivation of a closed-form upper bound for transient length using linearized dynamics for spectral radius ρ<1.
• Ensemble experiments to analyze transient-time distributions for spectral radius near and above unity.
• Analysis of network topology by interpreting the recurrent weight matrix as a directed graph to understand activity persistence.

Main Results:

• For ρ<1, a theoretical bound on transient length is established, dependent on tolerance and initial separation.
• For ρ≥1, heavy-tailed transient-time distributions are observed, with a significant fraction requiring extended washout.
• Non-linear saturation effectively shortens transients and can restore the Echo State Property, influenced by input amplitude and leakage.
• Network topology dictates activity persistence: cycle-rich structures and downstream nodes sustain activity, while others become quiescent.

Conclusions:

• Washout in reservoir computing is a quantifiable and engineerable property, not merely a heuristic.
• Network topology analysis provides insights into activity dynamics and guides reservoir design.
• Practical design rules include pre-screening reservoirs, favoring cycle-rich topologies, and tuning input/leakage parameters for robust and data-efficient RC.