Using shrinkage methods to estimate treatment effects in overlapping subgroups in randomized clinical trials with a...

In randomized controlled trials, forest plots are frequently used to investigate the homogeneity of treatment effect estimates in pre-defined subgroups. However, the interpretation of subgroup-specific treatment effect estimates requires great care due to the smaller sample size of subgroups and the large number of investigated subgroups. Bayesian shrinkage methods have been proposed to address these issues, but they often focus on disjoint subgroups while subgroups displayed in forest plots are overlapping, i.e., each subject appears in multiple subgroups. In our proposed approach, we first build a flexible Cox model based on all available observations, including treatment-by-subgroup interaction terms for all subgroups of interest. We explore penalized partial likelihood estimation with lasso or ridge penalties for interaction terms, and Bayesian estimation with a regularized horseshoe prior. In a second step, the Cox model is marginalized to obtain treatment effect estimates for all subgroups. We illustrate these methods using data from a randomized clinical trial in follicular lymphoma and evaluate their properties in a simulation study. In all simulation scenarios, the overall mean-squared error is substantially smaller for penalized and shrinkage estimators compared to the standard subgroup-specific treatment effect estimator but leads to some bias for heterogeneous subgroups. A naive overall sample estimator also outperforms the standard subgroup-specific estimator in terms of the overall mean-squared error for all scenarios except for a scenario with substantial heterogeneity. We recommend that subgroup-specific estimators are routinely complemented by treatment effect estimators based on shrinkage methods. The proposed methods are implemented in the R package bonsaiforest.