A Joint Time-Frequency Domain Transformer for multivariate time series forecasting.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Time Series Analysis

Background:

• Transformer models excel in sequence modeling but face computational challenges with long sequences.
• Multivariate forecasting requires capturing complex temporal dependencies and non-stationarity.

Purpose of the Study:

• To introduce a novel Transformer architecture, the Joint Time-Frequency Domain Transformer (JTFT), for efficient long-term multivariate forecasting.
• To enhance predictive performance while reducing computational demands.

Main Methods:

• JTFT integrates time and frequency domain representations for prediction.
• Frequency domain captures multi-scale dependencies using learnable frequencies.
• Time domain uses recent data points to model local relationships and non-stationarity.
• A low-rank attention layer efficiently handles cross-dimensional dependencies.

Main Results:

• JTFT achieves linear computational complexity, independent of input sequence length.
• The model demonstrates superior predictive performance compared to state-of-the-art baselines.
• Experiments conducted on eight diverse real-world datasets validated JTFT's effectiveness.

Conclusions:

• JTFT offers an efficient and high-performing solution for long-term multivariate forecasting.
• The joint time-frequency approach effectively addresses limitations of traditional Transformer models.