GBT: Two-stage transformer framework for non-stationary time series forecasting

Area of Science:

• * Artificial Intelligence
• * Machine Learning
• * Data Science

Background:

• * Time series forecasting Transformer (TSFT) models often overfit, particularly with non-stationary data, due to poor initialization of unknown decoder inputs.
• * Existing methods struggle with the differing statistical properties between input and prediction sequences in forecasting tasks.

Purpose of the Study:

• * To address the over-fitting and statistical property mismatch issues in TSFT for improved time series forecasting.
• * To introduce a novel framework, GBT (Good Beginning Transformer), that enhances forecasting performance.

Main Methods:

• * Proposed GBT, a two-stage Transformer framework decoupling prediction into Auto-Regression and Self-Regression stages.
• * Introduced a 'Good Beginning' concept where Auto-Regression outputs initialize the Self-Regression stage.
• * Developed an Error Score Modification module to boost Self-Regression stage forecasting capability.

Main Results:

• * GBT significantly outperforms state-of-the-art TSFT models (FEDformer, Pyraformer, ETSformer) and other forecasting models (SCINet, N-HiTS) on seven benchmark datasets.
• * GBT achieves superior performance with canonical attention and convolution, demonstrating lower time and space complexity.
• * The GBT framework proved generalizable and capable of enhancing other forecasting models.

Conclusions:

• * GBT effectively mitigates over-fitting in TSFT by providing a better input initialization through its two-stage approach.
• * The proposed framework offers a more efficient and accurate solution for time series forecasting, outperforming existing methods.
• * GBT's modular design allows for integration with other models to improve their forecasting capabilities.