Chain-Structure Echo State Network With Stochastic Optimization: Methodology and Application.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Multivariate time series prediction is crucial for many scientific and engineering fields.
• Existing deep recurrent neural networks face challenges in handling complex, high-dimensional data.
• A novel approach is needed to improve the accuracy and robustness of time series forecasting models.

Purpose of the Study:

• To propose a new deep recurrent neural network architecture, the chain-structure echo state network (CESN).
• To develop an effective training mechanism combining least-squares regression and stochastic local search (SLS) for CESN.
• To evaluate the performance and robustness of the proposed CESN model in various prediction tasks.

Main Methods:

• The proposed CESN utilizes stacked subnetwork modules and a "divide and conquer" strategy.
• Input vectors are clustered, and CESN integrates predictions from each clustered variable.
• A hybrid training approach uses least-squares regression for pretraining and SLS for fine-tuning weights to global optima, with early stopping to prevent overfitting.

Main Results:

• The CESN architecture demonstrates effectiveness in multivariate time series prediction.
• The SLS-CESN training method significantly reduces the loss function and avoids overfitting.
• Evaluations on chaos prediction benchmarks and real-world applications confirm the model's robustness and accuracy.

Conclusions:

• The proposed chain-structure echo state network (CESN) is a powerful deep recurrent neural network for multivariate time series prediction.
• The combination of data clustering and stacked subnetworks provides a robust framework for complex forecasting.
• The SLS-CESN training strategy ensures efficient optimization and reliable performance, validated across diverse applications.