Causal Mask in Transformer via Transfer Entropy Estimation from Vector Autoregressive Learning for Multivariate Time...

Area of Science:

• Data Science
• Machine Learning
• Time Series Analysis

Background:

• Time series forecasting is complex due to spurious correlations in finance and climate science.
• Existing methods struggle to disentangle autocorrelation from cross-variable causal effects.

Purpose of the Study:

• To propose ARCausal, a novel forecasting framework combining transfer entropy-based causal discovery and Transformer attention.
• To improve predictive performance and interpretability in time series forecasting.

Main Methods:

• Integrating transfer entropy (TE) for causal discovery with Transformer attention modeling.
• Developing a sparse causal masking mechanism derived from TE and refined using vector autoregression (VAR).
• The mask suppresses noninformative dependencies and differentiates autocorrelation from cross-variable causal effects.

Main Results:

• Consistent improvements over strong baselines across nine benchmark datasets.
• Achieved up to [Formula: see text] reduction in Mean Squared Error (MSE).
• Demonstrated enhanced interpretability of learned causal structures through visualizations.

Conclusions:

• ARCausal effectively captures dynamic causal interactions for improved time series forecasting.
• The framework offers a computationally efficient and interpretable solution for complex forecasting tasks.
• Publicly available code facilitates further research and application.