Delayformer: Spatiotemporal Transformation for Predicting High-Dimensional Dynamics.

Area of Science:

• Dynamical Systems
• Machine Learning
• Time Series Analysis

Background:

• Accurate time series prediction is crucial across scientific fields.
• High-dimensional systems present challenges due to nonlinearity and complex variable interactions, especially with limited, noisy data.

Purpose of the Study:

• Introduce the Delayformer framework for simultaneous prediction of all variables in high-dimensional time series.
• Develop a novel multivariate spatiotemporal information (mvSTI) transformation to address prediction challenges.

Main Methods:

• Utilize delay embedding theory to transform observed variables into delay-embedded states (vectors).
• Employ a shared Visual Transformer (ViT) encoder for cross-representation of dynamical states.
• Implement distinct linear decoders for parallel prediction of next states, effectively forecasting all original variables.

Main Results:

• Delayformer demonstrates superior performance over state-of-the-art methods on synthetic and real-world datasets.
• The framework successfully overcomes nonlinearity and cross-interaction problems by predicting system states.
• Achieved high accuracy in forecasting tasks, outperforming existing approaches.

Conclusions:

• Delayformer offers a robust solution for multivariate time series prediction, even with limited and noisy data.
• The model's ability to predict system states provides theoretical and computational advantages.
• Demonstrated broad applicability across various scenarios through cross-domain forecasting tasks.