Probingformer: Modeling cross-dimension dependencies via pretraining-probing for long-term time series forecasting

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Multivariate time series forecasting (MTSF) advances often use supervised learning and pretraining-fine-tuning.
• Pretraining-probing approaches in MTSF have been underexplored due to historically weaker empirical results.
• Channel-dependent methods in MTSF can overfit to spurious correlations.

Purpose of the Study:

• Revisit pretraining-probing as a principled method to decouple representation learning from forecasting in MTSF.
• Introduce Probingformer, a framework to enhance MTSF performance by explicitly capturing variable correlations.
• Mitigate overfitting to spurious correlations prevalent in MTSF.

Main Methods:

• Leverage pretraining to learn rich temporal and variable interactions.
• Employ a lightweight probing step for forecasting after representation learning.
• Introduce Cross-Temporal Dependency Module (TDM) for intra-sequence patterns and Cross-Dimension Dependency Module (DDM) for inter-variable correlations.

Main Results:

• Probingformer achieves state-of-the-art performance across ten diverse MTSF datasets.
• The framework effectively captures inter-variable correlations.
• Probingformer demonstrates mitigation of overfitting, even as a channel-dependent method.

Conclusions:

• Pretraining-probing is a viable and effective paradigm for MTSF.
• Probingformer offers a robust solution for accurate and reliable multivariate time series forecasting.
• The proposed TDM and DDM modules enhance structural modeling for improved MTSF performance.