DRCNN: decomposing residual convolutional neural networks for time series forecasting

Area of Science:

• Artificial Intelligence
• Machine Learning
• Time Series Analysis

Background:

• Transformer models demonstrate strong performance in long-term time series forecasting by capturing extended dependencies.
• Limitations of Transformers include poor performance on small datasets due to insufficient inductive bias and lack of focus on sequence continuity, hindering short and long-term forecasting.
• Existing models often overlook the importance of data continuity in time series forecasting.

Purpose of the Study:

• To address the limitations of Transformer models in time series forecasting, particularly on small datasets and for capturing sequence continuity.
• To propose an efficient Transformer-based design that enhances forecasting accuracy.
• To leverage the strengths of convolutional neural networks and Transformer architectures for improved time series prediction.

Main Methods:

• A Decomposing Residual Convolutional Neural Network (DRCNN) is proposed, which decomposes time series data into residual and trend components.
• A specialized convolution block (DR-Block) is designed to extract features, inspired by the structural design of Transformers.
• A Multi-head Sequence method, mimicking the multi-head attention in Transformers, is introduced to process longer input sequences for more accurate forecasts.

Main Results:

• The proposed DRCNN model effectively utilizes data continuity by processing decomposed residual and trend terms.
• The DR-Block demonstrates strength in feature extraction, aligning with Transformer architectural principles.
• The Multi-head Sequence method enables the network to handle longer inputs, leading to improved forecasting accuracy.
• The model achieved state-of-the-art performance across several benchmark datasets.

Conclusions:

• The DRCNN, combined with the Multi-head Sequence method, offers an effective solution for time series forecasting, overcoming limitations of standard Transformers.
• The proposed approach enhances the ability to capture both short-term and long-term dependencies by focusing on data continuity.
• The model's state-of-the-art performance validates its efficacy for diverse time series forecasting tasks.