STTRE: A Spatio-Temporal Transformer with Relative Embeddings for multivariate time series forecasting.

Area of Science:

• Data Science
• Artificial Intelligence
• Machine Learning

Background:

• Multivariate time series data is increasingly prevalent across various scientific disciplines.
• Existing Transformer-based models often overlook spatial components and use flawed temporal embeddings for time series analysis.
• This leads to suboptimal accuracy in multivariate time series forecasting.

Purpose of the Study:

• To propose a novel Transformer-based framework, the Spatio-Temporal Transformer with Relative Embeddings (STTRE), for enhanced multivariate time series forecasting.
• To address the limitations of current models in capturing spatio-temporal dependencies.
• To improve the accuracy and effectiveness of time series analysis.

Main Methods:

• Developed the Spatio-Temporal Transformer with Relative Embeddings (STTRE), adapting the Transformer architecture for time series.
• Redesigned relative position representations into relative embeddings to better detect spatial, temporal, and spatio-temporal dependencies.
• Restructured the multi-head attention mechanism to fully leverage the proposed relative embeddings.

Main Results:

• The STTRE model effectively captures latent spatio-temporal dependencies.
• Achieved up to a 24% improvement in forecasting accuracy compared to state-of-the-art models.
• Demonstrated superior performance on a diverse set of publicly available multivariate time series forecasting datasets.

Conclusions:

• The proposed STTRE framework offers a significant advancement in multivariate time series forecasting.
• Effective utilization of spatio-temporality through relative embeddings is crucial for improving model accuracy.
• STTRE provides a more robust and accurate solution for complex time series analysis.