Spatiotemporal Transformer Neural Network for Time-Series Forecasting.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• High-dimensional, short-term time-series prediction is challenging due to limited data and the curse of dimensionality.
• Existing methods struggle with accuracy and robustness in complex, multi-variate forecasting tasks.

Purpose of the Study:

• To propose a novel spatiotemporal transformer neural network (STNN) for efficient and accurate multi-step-ahead prediction of high-dimensional short-term time-series.
• To enhance prediction accuracy by leveraging spatial information and a continuous attention mechanism.
• To demonstrate the STNN's capability in reconstructing dynamical system phase spaces.

Main Methods:

• Developed a spatiotemporal transformer neural network (STNN) incorporating continuous spatial self-attention, temporal self-attention, and transformation attention mechanisms.
• Utilized the spatiotemporal information (STI) transformation equation to exploit high-dimensional spatial information.
• Employed a continuous attention mechanism for improved prediction accuracy.

Main Results:

• The STNN model accurately and robustly predicts high-dimensional short-term time-series in a multi-step-ahead manner.
• Continuous attention mechanisms led to more accurate predictions compared to previous studies.
• The STNN demonstrated the ability to reconstruct the phase space of dynamical systems.
• Experimental results showed significant outperformance of the STNN over existing methods on various benchmarks and real-world systems.

Conclusions:

• The proposed STNN is an effective model for high-dimensional, short-term time-series prediction, outperforming existing approaches.
• The integration of spatial and temporal attention mechanisms is crucial for capturing complex dynamics.
• The STNN offers a robust solution for multi-step-ahead forecasting in diverse applications.