Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex...

Area of Science:

• Computational physics
• Machine learning
• Dynamical systems

Background:

• Recurrent Neural Networks (RNNs) are powerful tools for analyzing complex spatiotemporal dynamics.
• Reservoir Computing (RC) and Backpropagation Through Time (BPTT) are key training methods for RNNs.
• Understanding their comparative efficiency in forecasting is crucial for complex system analysis.

Purpose of the Study:

• To evaluate the forecasting efficiency of RC and BPTT for RNNs in high-dimensional and reduced-order complex systems.
• To compare the predictive accuracy and long-term statistical capture of these methods using benchmark chaotic systems.
• To analyze the implementation and limitations of each method, especially concerning parallel computing and data availability.

Main Methods:

• Utilized gated network architectures for RNNs, training with both RC and BPTT.
• Employed the Lorenz-96 and Kuramoto-Sivashinsky (KS) equations as benchmark chaotic systems.
• Quantified prediction accuracy, long-term statistics, training time, and Lyapunov Spectrum for the KS equation.

Main Results:

• RC demonstrated superior predictive performance and statistical capture with full state data, requiring less training time.
• BPTT-trained RNNs showed better forecasting for reduced-order systems, avoiding divergence issues seen in large-scale RC models.
• BPTT achieved comparable accuracy to RC in quantifying the Lyapunov Spectrum of the KS equation, a novel finding.

Conclusions:

• RNNs, trained via either RC or BPTT, represent a potent computational framework for learning and forecasting complex spatiotemporal systems.
• The choice between RC and BPTT depends on data availability (full vs. reduced order) and system complexity.
• This study provides quantitative insights into the strengths and weaknesses of RC and BPTT for complex system forecasting.