Continual Slow-and-Fast Adaptation of Latent Neural Dynamics (CoSFan): Meta-Learning What-How & When to Adapt

Area of Science:

• Machine Learning
• Artificial Intelligence
• Time-Series Analysis

Background:

• Forecasting high-dimensional time-series requires adapting to changing system dynamics.
• Stationary data distributions are often unavailable, and retraining models can cause catastrophic forgetting.
• Continual learning methods struggle with adapting to diverse and shifting dynamics.

Purpose of the Study:

• To develop a continual meta-learning (CML) framework for adaptive time-series forecasting.
• To enable both fast adaptation to current dynamics and slow updates to retain past knowledge.
• To address the challenge of non-stationary data distributions in complex systems.

Main Methods:

• Introduced a continual meta-learning framework (CoSFan) for continual slow-and fast adaptation.
• Utilized a feed-forward meta-model to infer system identity and adaptation strategies.
• Developed novel strategies for detecting data distribution shifts and identifying dynamics.
• Incorporated fixed-memory experience replay for continual meta-model updates.

Main Results:

• The meta-learning and continual learning components were crucial for forecasting across non-stationary distributions.
• The CoSFan framework demonstrated superior performance compared to existing CML alternatives.
• The feed-forward meta-model and task-aware continual learning strategies proved effective.

Conclusions:

• CoSFan effectively enables continual slow-and fast adaptation for time-series forecasting in dynamic systems.
• The proposed framework overcomes catastrophic forgetting and adapts to diverse, non-stationary data.
• This approach offers a significant advancement in continual meta-learning for complex forecasting tasks.