Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir...

Area of Science:

• Dynamical Systems and Chaos Theory
• Machine Learning and Artificial Intelligence
• Computational Physics

Background:

• Machine learning (ML) models can predict chaotic system dynamics.
• Feedback loops in ML models can lead to instability and rapid error growth.
• Adding noise during training is a known technique to mitigate instability.

Purpose of the Study:

• To develop a new regularization technique for ML models with memory.
• To deterministically approximate the effect of input noise during training.
• To evaluate the effectiveness of this new technique, Linearized Multi-Noise Training (LMNT), against existing methods.

Main Methods:

• Formulated a new penalty term in the loss function for ML models.
• Developed Linearized Multi-Noise Training (LMNT) to approximate noise effects.
• Applied LMNT and other regularization techniques to reservoir computing models for the Kuramoto-Sivashinsky equation.

Main Results:

• LMNT and input noise regularization yield indefinitely stable climate predictions.
• Both methods produce climate statistics closely matching the true chaotic system.
• Short-term forecasts are significantly more accurate with LMNT and noise regularization compared to other techniques.

Conclusions:

• LMNT effectively stabilizes ML models for chaotic system prediction.
• The deterministic nature of LMNT allows for rapid hyperparameter tuning.
• LMNT offers a promising approach for accurate and stable long-term forecasting of chaotic dynamics.