Predicting complex time series with deep echo state networks

Area of Science:

• Computational neuroscience
• Machine learning
• Time series analysis

Background:

• Real-world time series forecasting is challenging due to data complexity.
• Echo state networks (ESNs), a type of recurrent neural network, offer efficient training for forecasting.
• Deep ESNs, with stacked reservoir layers, aim to capture more complex dynamics but are less studied.

Purpose of the Study:

• To analyze the performance of deep echo state networks (ESNs) for time series forecasting.
• To evaluate variations in deep ESN network structures, including hybrid models.
• To compare deep ESNs against baseline ESNs and flat hybrid ESNs.

Main Methods:

• Investigated deep echo state networks (ESNs) with stacked reservoir layers.
• Implemented and tested hybrid deep ESNs integrating knowledge-based models.
• Compared prediction accuracy and error reduction across different network configurations using Mackey-Glass and zebrafish cardiac data.

Main Results:

• Deep ESNs improved prediction accuracy by up to 65% on chaotic data and 14% on experimental data compared to baseline ESNs.
• Deep hybrid ESNs reduced error by up to 59% on chaotic data and 11% on experimental data versus flat hybrid ESNs.
• The hybrid approach benefited experimental data, and deep structures enhanced prediction robustness.

Conclusions:

• Deep ESNs offer significant improvements in time series forecasting accuracy and robustness.
• Hybrid deep ESNs provide a powerful approach, especially for complex, real-world datasets.
• Network structure variations, particularly depth and hybrid integration, are crucial for optimizing forecasting performance.