Improving policy-oriented agent-based modeling with history matching: A case study

Advances in computing power and data availability have led to growing sophistication in mechanistic mathematical models of social dynamics. Increasingly these models are used to inform real-world policy decision-making, often with significant time sensitivity. One such modeling approach is agent-based modeling, which offers particular strengths for capturing spatial and behavioral realism, and for in-silico experiments (varying input parameters and assumptions to explore their downstream impact on key outcomes). To be useful in the real-world, these models must be able to qualitatively or quantitatively capture observed empirical phenomena, forming the starting point for subsequent experimentation. One recent example is the COVID-19 pandemic, where epidemiological agent-based models informed policy and response planning worldwide. Throughout, modeling teams often had to spend valuable time and effort aligning their models to data, also known as calibration. Since many agent-based models are computationally intensive, the calibration process constrains the questions and scenarios policymakers may explore in time-sensitive situations. In this paper, we combine history matching, heteroskedastic Gaussian process modeling, and approximate Bayesian computation to address this bottleneck, substantially increasing efficiency and thus widening the range of utility for policy models. We illustrate our approach with a case study using a previously published and widely used epidemiological model, the Covasim model.