Physics-based models outperform AI weather forecasts of record-breaking extremes

Area of Science:

• Meteorology
• Artificial Intelligence
• Climate Science

Background:

• Artificial intelligence (AI) models show promise in weather forecasting, exceeding traditional numerical models on benchmarks.
• The reliability of AI in predicting unprecedented extreme weather events remains uncertain.

Purpose of the Study:

• To compare the performance of state-of-the-art AI weather models against a physics-based model for record-breaking extreme events.
• To identify current limitations of AI models in forecasting extreme weather phenomena.

Main Methods:

• Comparative analysis of AI models (GraphCast, Pangu-Weather, Fuxi) and the High Resolution Forecast (HRES) physics-based model.
• Evaluation of forecast errors for record-breaking heat, cold, and wind events across various lead times.

Main Results:

• The HRES physics-based model consistently outperformed AI models for record-breaking weather extremes.
• AI models exhibited larger forecast errors for extreme events compared to HRES.
• AI models tended to underestimate the frequency and intensity of record-breaking events, with specific biases for hot and cold records.

Conclusions:

• Current AI weather models have limitations in extrapolating beyond training data for extreme events.
• Physics-based models remain superior for forecasting high-impact, record-breaking weather, crucial in a warming climate.
• Further development and verification of AI models are necessary for high-stakes applications like early warning systems.