TS-PET: A Novel Framework for Fine-Tuning Pretrained Time-Series Models

Area of Science:

• Artificial Intelligence
• Machine Learning
• Time Series Analysis

Background:

• Foundation models for time series offer strong zero-shot abilities.
• Existing adaptation methods face challenges like inefficient fine-tuning and overfitting due to computational constraints and parameter-heavy prediction heads.

Purpose of the Study:

• To propose a novel fine-tuning framework, Time Series Parameter-Efficient Transformer (TS-PET), for efficient adaptation of time series foundation models.
• To address the limitations of current adaptation techniques, specifically concerning computational efficiency and overfitting.

Main Methods:

• Developed a lightweight prediction module reducing parameters by over 80% to mitigate overfitting.
• Implemented specialized pruned low-rank adaptation (LoRA) with robust rank allocation for improved efficiency.
• Conducted extensive experiments on eight diverse benchmarks.

Main Results:

• TS-PET achieved state-of-the-art accuracy, outperforming existing methods like MOMENT, PatchTST, and adaptive LoRA variants.
• Demonstrated superior parameter efficiency compared to current adaptation techniques.
• Successfully enabled scalable adaptation without performance compromise.

Conclusions:

• TS-PET offers a highly efficient and accurate solution for adapting time series foundation models.
• The proposed framework effectively tackles challenges in fine-tuning and parameter efficiency.
• TS-PET facilitates broader adoption and application of foundation models in time series analysis.