Online sequential echo state network with sparse RLS algorithm for time series prediction.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Time Series Analysis

Background:

• Echo State Networks (ESNs) are prevalent for time series prediction.
• Overfitting and network size control are critical challenges in practical ESN applications.

Purpose of the Study:

• To propose a novel algorithm, the online sequential ESN with sparse recursive least squares (OSESN-SRLS), addressing overfitting and network size.
• To enhance the applicability of ESNs in real-world scenarios.

Main Methods:

• Utilized ℓ0 and ℓ1 norm sparsity penalty constraints on output weights for network size management.
• Combined the sparse recursive least squares (SRLS) algorithm with subgradients for output weight matrix estimation.
• Developed an adaptive mechanism for selecting the ℓ0 or ℓ1 norm regularization parameter.

Main Results:

• The developed SRLS algorithm demonstrated performance comparable to or better than regular Recursive Least Squares (RLS).
• Theoretical analysis confirmed the convergence and effectiveness of the OSESN-SRLS algorithm.
• Simulation results showed OSESN-SRLS consistently outperformed other ESNs in estimation accuracy and network compactness.

Conclusions:

• The proposed OSESN-SRLS algorithm effectively mitigates overfitting and controls network size in ESNs.
• OSESN-SRLS offers significant improvements in prediction accuracy and model compactness for time series tasks.
• The theoretical guarantees and simulation results validate the robustness and efficacy of the OSESN-SRLS approach.