Recurrent neural networks and robust time series prediction

Area of Science:

• Machine Learning
• Artificial Intelligence
• Time Series Analysis

Background:

• Recurrent neural networks (RNNs) are powerful tools for time series analysis.
• Conventional training methods like least squares are sensitive to outliers in data.
• This sensitivity can degrade the predictive performance of RNNs.

Purpose of the Study:

• To propose and evaluate a robust learning algorithm for recurrent neural networks.
• To demonstrate the limitations of least squares estimation in the presence of outliers.
• To enhance the predictive accuracy of RNNs for time series data.

Main Methods:

• A novel robust learning algorithm based on soft outlier filtering is introduced.
• The algorithm filters outliers from both the target function and inputs of the neural network.
• The performance of robustly trained networks is compared against least squares trained networks using synthetic and real-world datasets.

Main Results:

• Least squares estimated recurrent neural networks exhibit a lack of robustness towards outliers.
• The proposed filtering method effectively removes outliers, including those from holidays in the Puget Power Electric Demand time series.
• Networks trained with the robust algorithm show improved prediction accuracy compared to those trained on unfiltered data.

Conclusions:

• Robust recurrent neural networks, specifically NARMA(p,q) models, offer advantages for time series with moving average components.
• The proposed soft outlier filtering technique enhances the reliability and predictive power of recurrent neural networks.
• This robust approach is crucial for accurate time series forecasting in the presence of noisy data.