Causal Intervention Is What Large Language Models Need for Spatio-Temporal Forecasting.

Area of Science:

• Artificial Intelligence
• Data Science
• Urban Planning

Background:

• Spatio-temporal forecasting is vital for smart city applications like traffic and energy prediction.
• Sparse data and model limitations (spurious correlations, hallucinations) hinder accurate forecasting.
• Existing methods struggle with cross spatio-temporal and cross-scale feature inference.

Purpose of the Study:

• To introduce a novel model, STCInterLLM, for accurate spatio-temporal forecasting with sparse data.
• To address spatial spurious associations and LLM hallucinations in forecasting models.
• To improve the learning and inference of cross spatio-temporal and cross-scale features.

Main Methods:

• Developed a spatio-temporal causal intervention large language model (STCInterLLM).
• Employed a causal intervention encoder to update adaptive graphs and mitigate spatial spurious correlations.
• Utilized chain-of-action prompting to decompose predictions, enhance causal representations, and reduce LLM hallucinations.
• Integrated a marker alignment module for consistency across model components.

Main Results:

• STCInterLLM demonstrates state-of-the-art performance in spatio-temporal forecasting.
• The model effectively handles sparse data and improves prediction accuracy.
• Experiments on power and transportation systems confirm consistent performance across diverse scenarios.

Conclusions:

• STCInterLLM offers a robust solution for spatio-temporal forecasting challenges, particularly with sparse data.
• The proposed causal intervention and prompting strategies effectively address model limitations.
• The model enables accurate forecasting of complex spatio-temporal evolution patterns.