Learning from mistakes-Assessing the performance and uncertainty in process-based models.

Area of Science:

• Environmental modeling
• Hydrology
• Machine learning applications

Background:

• Process-based models are crucial for understanding environmental systems and decision-making.
• Model errors can arise from simplifications, misrepresentations, or missing processes.
• Systematic analysis of model errors is essential for improving model accuracy and reliability.

Purpose of the Study:

• To develop and evaluate a workflow for analyzing process-based model errors.
• To link model errors to specific process representations.
• To enhance process understanding and predictive capabilities of environmental models.

Main Methods:

• A three-step approach combining machine learning (ML) error modeling, SHapley Additive exPlanations (SHAP) for local error attribution, and clustering of SHAP values.
• Training an ML model to predict errors of a process-based model using input data and other variables.
• Utilizing SHAP and principal component analysis to explain individual error predictions and clustering SHAP values to identify error patterns.

Main Results:

• The ML error model successfully predicted residuals of the HFLUX stream water temperature model.
• Clustering of SHAP values revealed three distinct error groups.
• Identified error groups were primarily associated with shading and vegetation-emitted longwave radiation.

Conclusions:

• Model errors are often informative and not random, providing insights into model deficiencies.
• The proposed workflow effectively links model errors to specific processes, facilitating targeted improvements.
• This approach enhances trust in model components and guides future model development for environmental applications.