Robust and Adaptive Online Time Series Prediction with Long Short-Term Memory

Area of Science:

• Machine Learning
• Data Science
• Time Series Analysis

Background:

• Online time series prediction is crucial across diverse fields like finance and signal processing.
• Real-world data frequently contains outliers that can significantly degrade prediction model performance.
• Existing methods often struggle to maintain accuracy when faced with noisy, outlier-prone time series data.

Purpose of the Study:

• To develop a robust and adaptive online gradient learning method for Long Short-Term Memory (LSTM) networks.
• To address the challenge of outliers in time series prediction.
• To enhance the accuracy and reliability of online time series forecasting models.

Main Methods:

• Proposes RoAdam (Robust Adam), an adaptive online gradient learning method for LSTMs.
• Modifies the Adam optimizer to dynamically adjust learning rates based on relative prediction errors.
• Implements a weighted average tracking of the loss function's relative prediction error.

Main Results:

• RoAdam demonstrates superior performance in time series prediction compared to existing LSTM-based methods.
• The adaptive learning rate mechanism effectively reduces the adverse impact of outliers.
• Experiments on both synthetic and real-world time series validate the method's efficacy.

Conclusions:

• RoAdam offers a robust solution for online time series prediction in the presence of outliers.
• The adaptive learning rate adjustment is key to handling noisy data.
• This method improves the reliability of LSTM-based predictions for practical applications.