Machine learning for predicting chaotic systems

Area of Science:

• Complex Systems Science
• Computational Physics
• Data Science

Background:

• Predicting chaotic dynamical systems is crucial for fields like weather forecasting but is difficult due to sensitive dependence on initial conditions.
• Traditional modeling requires domain expertise, prompting a move towards data-driven machine learning (ML) approaches.
• Current research offers no consensus on the optimal ML methods for chaotic system prediction.

Purpose of the Study:

• To compare various lightweight and heavyweight ML architectures for predicting chaotic dynamical systems.
• To evaluate ML performance using established benchmark datasets and a new dataset incorporating uncertainty quantification.
• To introduce a novel performance metric, the cumulative maximum error, specifically designed for chaotic systems.

Main Methods:

• Extensive comparison of diverse ML architectures (lightweight and heavyweight) on benchmark datasets.
• Introduction of new variants of existing ML methods and tailored hyperparameter tuning based on computational cost.
• Development and application of the cumulative maximum error metric for evaluating prediction accuracy in chaotic systems.

Main Results:

• Well-tuned simple ML models and even untuned baselines frequently outperformed state-of-the-art deep learning models.
• Model performance demonstrated significant variability depending on the experimental setup and data characteristics.
• The newly introduced cumulative maximum error metric proved effective in evaluating chaotic system predictions.

Conclusions:

• The choice of ML method should be carefully aligned with the specific data characteristics of the chaotic system.
• Overly complex deep learning models are not always superior and can be outperformed by simpler, well-tuned methods.
• Indiscriminate application of complex models without considering data properties and computational cost is discouraged.