Sample-efficient fine-tuning with textual prompts for time series forecasting.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Time Series Analysis

Background:

• Cross-domain fine-tuning of time series models presents challenges like high training costs and poor adaptability.
• Existing methods struggle to efficiently adapt models to new domains without extensive retraining.

Purpose of the Study:

• To propose a novel parameter-efficient adaptation method for time series forecasting.
• To leverage semantic information from textual prompts to guide numerical time series predictions.
• To enhance the adaptability and reduce the training costs of forecasting models.

Main Methods:

• Developed the Cue-driven Feature Fusion Network (CFF-Net) integrating semantic cues from text with numerical time series features.
• Employed a Semantic Prompt Encoding Module (SPM) to convert numerical sequences into natural language descriptions for GPT-2 processing.
• Utilized a Dynamic Semantic Modulation Module (DSM) to map semantic representations to scaling and shifting factors for modulating predictions via Scale-and-Shift Feature (SSF).

Main Results:

• CFF-Net demonstrated reduced Mean Squared Error (MSE) compared to PatchTST on Weather and TCTS datasets, particularly in low-data regimes (e.g., 12.50% MSE reduction on Weather with 30% samples).
• Improvements varied across datasets and metrics, indicating domain-specific performance.
• The model achieved better forecasting performance in several limited-data scenarios.

Conclusions:

• Semantic prompt guidance offers a viable strategy for enhancing time series forecasting performance, especially under data scarcity.
• CFF-Net provides a parameter-efficient approach to adaptation by fixing most backbone parameters.
• The integration of LLM-derived semantic cues shows promise for improving the adaptability of time series models.