DyGraphformer: Transformer combining dynamic spatio-temporal graph network for multivariate time series forecasting.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Time Series Analysis

Background:

• Transformer models excel at capturing long-term temporal dependencies in multivariate time series (MTS) forecasting via self-attention.
• However, effectively modeling spatial correlations across series in MTS remains a challenge for Transformers.
• Existing Graph Neural Network (GNN) methods assume static relationships, which is insufficient for time-varying spatial dependencies.

Purpose of the Study:

• To propose DyGraphformer, a novel architecture that integrates graph convolution into Transformer for improved MTS forecasting.
• To dynamically infer time-varying spatial dependencies by incorporating historical spatial information.
• To accelerate model execution by abandoning complex recursive decoder modules.

Main Methods:

• Input embedding using Dimension Segment Wise (DSW) with integrated positional and node-level embeddings.
• Time self-attention layer for temporal dependencies and a dynamic graph convolutional layer for spatial dependencies.
• Dynamic graph convolutional layer uses Gated Recurrent Unit (GRU) for historical spatial dependencies and infers graph structure in multiple subspaces.

Main Results:

• DyGraphformer effectively models both temporal and spatial dependencies in MTS.
• The dynamic graph convolutional layer successfully captures time-varying spatial relationships.
• Hierarchical encoder learning enhances spatio-temporal information utilization at different scales.

Conclusions:

• DyGraphformer significantly outperforms state-of-the-art Transformer-based and GNN-based methods on seven real-world MTS datasets.
• The proposed model demonstrates superior performance in capturing complex spatio-temporal dynamics.
• DyGraphformer offers a promising advancement for accurate and efficient MTS forecasting.