Using QALYs as an Outcome for Assessing Global Prediction Accuracy in Diabetes Simulation Models

Area of Science:

• Health economics
• Biostatistics
• Medical simulation

Background:

• Diabetes simulation models are typically validated by predicting individual clinical events.
• Quality-adjusted life-years (QALYs) integrate mortality and clinical events for decision-making.
• Existing metrics may not fully capture overall model performance for economic evaluation.

Purpose of the Study:

• To demonstrate QALYs as an outcome measure for comparing simulation model performance.
• To identify the most accurate simulation model for health technology assessment and economic evaluation.
• To explore the utility of Q², the proportional reduction in error, as a model performance metric.

Main Methods:

• Simulated 14,729 EXSCEL trial participants using UK Prospective Diabetes Study Outcomes Model versions 1 and 2 (UKPDS-OM1 and UKPDS-OM2).
• Estimated QALYs over the trial period using observed events and survival data.
• Compared QALY predictions from UKPDS-OM1 and UKPDS-OM2 with observed QALYs and evaluated Q² against other metrics (MSE, MAE, bias, R²).

Main Results:

• UKPDS-OM2 demonstrated more accurate QALY predictions than UKPDS-OM1 (Q²: 0.822 vs. 0.786; MSE: 0.210 vs. 0.253).
• UKPDS-OM2 showed improved accuracy for mortality, myocardial infarction, and stroke predictions.
• Q² proved useful for comparing global prediction accuracy and identifying biased predictors, unlike R².

Conclusions:

• Q² is a valuable metric for comparing the overall predictive accuracy of diabetes simulation models.
• The Q² metric can aid in selecting the most accurate simulation models for economic evaluations and health technology assessment.
• The proposed methodology using Q² for QALYs can be extended to other disease areas for model evaluation and recalibration.