How Modelers Model: the Overlooked Social and Human Dimensions in Model Intercomparison Studies

Area of Science:

• Environmental modeling
• Computational science
• Ecology

Background:

• Model ensemble studies are vital for understanding complex environmental systems.
• Uncertainty in these studies often stems from modeler experience and calibration approaches, not just model complexity.
• Human and social attributes are increasingly recognized as critical factors in modeling outcomes.

Purpose of the Study:

• To investigate the decision-making rationale of modelers during ensemble calibration using a multi-criteria approach.
• To identify how modeler experience and access to data influence the calibration of biogeochemical models.
• To assess the impact of subjective weighting and cognitive biases on model intercomparison studies.

Main Methods:

• Applied a multi-criteria decision-making method to analyze modeler rationale.
• Compared 12 process-based biogeochemical models across five successive calibration stages.
• Analyzed the importance attributed to input variables by modelers based on model type and experience level.

Main Results:

• Modelers agreed on initial calibration variables but showed inconsistency in judging variable importance across stages.
• Subjective weighting of calibration data decreased as more variables were provided.
• Perceived importance of variables like fertilization, irrigation, soil properties, and initial carbon/nitrogen stocks differed by model type.
• Importance of variables like experimental duration and ecosystem exchange varied significantly with modeler experience.

Conclusions:

• Gradual data access across calibration stages can negatively influence interpretation consistency and introduce cognitive bias.
• Human factors, including modeler assumptions and perceptions of parameter importance, are as critical as numerical details for quality model calibration.
• Overlooking human and social attributes in modeling and model intercomparison studies leads to critical gaps in understanding outcomes.