Choosing a Metamodel of a Simulation Model for Uncertainty Quantification

Area of Science:

• Health economics
• Computational modeling
• Public health

Background:

• Metamodeling can significantly decrease computational costs for individual-level state transition simulation models (IL-STM).
• Metamodels are underutilized in health economics and public health due to a lack of guidance and accessible code.
• This study offers guidance on selecting metamodels for uncertainty quantification.

Purpose of the Study:

• To evaluate the prediction accuracy and computational efficiency of various metamodels for uncertainty quantification.
• To assess how metamodel performance varies with simulation model characteristics.
• To provide practical recommendations for choosing metamodels in health economic evaluations.

Main Methods:

• A simulation study was conducted using life-years gained (LYG) as the IL-STM outcome.
• Four metamodels were evaluated: linear models (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs).
• Metamodels were tested on a lung cancer IL-STM for probabilistic analysis.

Main Results:

• With low parameter uncertainty and sufficient simulation runs, all tested metamodels (LM, ANNs, GAMs, GP) achieved high accuracy (error <1% for LYG).
• Under higher parameter uncertainty, GP and ANN demonstrated superior prediction accuracy compared to LM.
• In the case study, the best-performing metamodel achieved a maximum error of approximately 2.1%.

Conclusions:

• For efficient and accurate uncertainty quantification, begin with linear models.
• If linear model accuracy is inadequate, progress to artificial neural networks and Gaussian process regression.
• These findings aid researchers in selecting appropriate metamodels for health economic simulation studies.