Calibration methods used in cancer simulation models and suggested reporting guidelines

Area of Science:

• Health economics and outcomes research
• Biostatistics and mathematical modeling
• Cancer prevention and control

Background:

• Computer simulation models are vital for economic and policy evaluation in cancer prevention and control.
• Model predictions, such as screening effectiveness, rely on accurate natural history parameter values.
• Calibration, the process of fitting model outputs to observed data, is crucial for determining these parameters.

Purpose of the Study:

• To catalogue the use and reporting of model calibration methods in cancer simulation literature.
• To identify best practices and areas for improvement in describing calibration procedures.
• To propose a standardized checklist to enhance the reporting of calibration methods in future studies.

Main Methods:

• Conducted a MEDLINE search (1980-2006) for cancer-screening models, supplemented by personal reference databases.
• Independently abstracted data on model type, parameter determination methods, and calibration protocols from 154 eligible articles.
• Classified models based on the reporting and methods used for calibration, distinguishing analytical models.

Main Results:

• Of 154 articles, 131 likely used calibration methods; explicit descriptions or references were found in 66% of these.
• Calibration target data were identified in nearly all articles reporting calibration methods.
• Methodological details like goodness-of-fit metrics were reported in 54% of calibration-focused articles, but algorithm details were scarce.

Conclusions:

• The use of cancer simulation modeling is growing, but comprehensive reporting of calibration procedures remains infrequent.
• Clear and detailed reporting of calibration is essential for the validity and credibility of simulation model outputs.
• A standardized Calibration Reporting Checklist is proposed to improve documentation and facilitate peer review of modeling methods.