Comparison of disease risk score methods to study treatment effect heterogeneity: a simulation study

Estimating treatment effects across disease risk scores (DRSs) is a common approach for assessing treatment effect heterogeneity in randomized trials. When external models are unavailable, the optimal approach for internally fitting the DRS model remains uncertain. This simulation study compares 3 internal derivation approaches, evaluating bias of estimated treatment effects within DRS-defined strata. We simulated trials under varying treatment effects (odds ratios [OR] of 1, 0.8, and 0.5) and treatment-covariate interactions. We fit DRS models on (1) a controls-only method, (2) the full sample ignoring treatment assignment, and (3) a random split-sample method of 50% of the controls, who were removed from the second stage of analysis. Additional simulations varied outcome incidence, sample size, randomization ratio, and the true DRS C-statistic. Bias decreased with the split-sample method (overall percent bias [OPB] 7.7% for OR = 0.8 with interactions) compared to the controls-only (OPB = 15.6%) and full-sample methods (OPB = 22.1%). Bias decreased more with the split-sample method than with controls-only and full-sample methods with larger sample sizes, higher outcome incidence, greater treated-to-control ratios, and larger C-statistics. These findings suggest split-sample methods may be the preferred approach to estimate treatment effect heterogeneity by the DRS in trials with sufficient data to support stable prediction modeling.