Granger-TSllm: Granger causality enhanced LLMs with residual-quantized tokenizer for multivariate time series...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Multivariate time series (MTS) forecasting is crucial for process monitoring and decision-making.
• Current deep learning models struggle with generalization due to training requirements.
• Large Language Models (LLMs) show promise for MTS forecasting but face modality gaps and dependency challenges.

Purpose of the Study:

• To propose a novel two-stage framework, Granger-TSllm, for enhanced MTS forecasting using LLMs.
• To address challenges in generating MTS representations for LLMs and capturing inter-variable dependencies.
• To improve the generalization capabilities of LLMs in MTS forecasting tasks.

Main Methods:

• Developed a Residual-Quantized Time Series Tokenizer for compact, discrete MTS embeddings.
• Utilized selective fine-tuning of LLMs for complex temporal modeling.
• Introduced a Granger Causality Modification Module to leverage inter-variable nonlinear correlations.

Main Results:

• Granger-TSllm significantly outperforms existing state-of-the-art MTS forecasting models.
• The framework demonstrates robust generalization capabilities.
• Achieved superior performance in few-shot and zero-shot forecasting scenarios.

Conclusions:

• Granger-TSllm offers a powerful and generalizable approach to MTS forecasting.
• The integration of LLMs with specialized modules enhances predictive accuracy.
• This method effectively bridges the modality gap for time series data in LLMs.