Multimodel ensembles improve predictions of crop-environment-management interactions

Area of Science:

• Agricultural Science
• Climate Change Modeling
• Statistical Analysis

Background:

• Crop multimodel ensembles (MMEs) are increasingly used to assess climate change impacts on agriculture.
• While ensemble means (e-mean) and medians (e-median) often show good predictive performance, their actual predictive quality and dependence on ensemble characteristics are not well understood.

Purpose of the Study:

• To evaluate the predictive quality of ensemble mean and median in crop modeling.
• To determine how ensemble characteristics influence the predictive quality of these ensemble predictors.

Main Methods:

• Empirical analysis using five MME studies on wheat, employing 25 crop models across diverse datasets and environments.
• Theoretical modeling of ensembles using four key parameters: average bias, model effect variance, environment effect variance, and interaction variance.
• Analysis of mean squared error (MSE) and mean squared error of prediction (MSEP) for ensemble predictors.

Main Results:

• Ensemble predictors (e-mean, e-median) demonstrate high skill, generally outperforming individual models across various environments and response variables.
• The mean squared error of the ensemble mean decreases with ensemble size when models are added randomly, but reaches a minimum with 2-6 models when best-fit models are prioritized.
• Analytical results show that the MSEP of the ensemble mean is consistently lower than the average MSEP of individual models and can be lower than the best individual model's MSEP under specific bias conditions.

Conclusions:

• Ensemble predictors offer significant advantages in predicting climate change impacts on agriculture, often surpassing individual model performance.
• The predictive quality of ensemble means is influenced by ensemble size and composition, with random addition leading to monotonic MSE decrease and prioritized addition showing a minimum at small ensemble sizes.
• It is crucial to evaluate the predictive quality of ensemble means for specific target populations of environments due to potential limitations and non-trivial minimum MSE values.