Choice of estimands and estimators affected the interpretation of results for some outcomes in a cluster-randomized...

In cluster- randomized trials (CRTs), different estimands can be targeted, such as the individual- or cluster-average effect. These two estimands can differ in magnitude when outcomes or treatment effects vary with cluster size (termed informative cluster size). When informative cluster size is present, commonly used estimators for CRTs, such as mixed-effects models and generalized estimating equations with an exchangeable correlation structure (termed GEEs(exch)), can be biased for both these estimands. With little documented evaluation of informative cluster size, it is currently unknown how commonly it occurs in practice. The aim of this work was to explore whether informative cluster size is present in a published cluster- randomized trial and to investigate its impact on trial results.

We reanalyzed the RESTORE CRT, which compared protocolized sedation with usual care for critically ill children. For each outcome, we first modeled the association between cluster size and outcome/treatment effect; next, we assessed the impact of informative cluster size by comparing differences between (1) individual-vs cluster-average estimates and (2) estimates from mixed-effects models and GEEs(exch) (which can be affected by informative cluster size) to those from independence estimating equations (IEEs) (which are robust to informative cluster size).

We found evidence of an association between cluster size and either outcomes or treatment effects for 16/33 outcomes (48%). This led to statistically significant differences between the individual- and cluster-average treatment effects for five of 33 outcomes (15%). There were >10% differences between (1) individual- and cluster-average treatment effect estimates for 17 outcomes (52%) and (2) estimates from mixed-effects models/GEEs(exch) and estimates from unweighted independence estimating equations for 12 outcomes (36%). For some outcomes, differences in the choice of estimator or estimand led to differences in the interpretation of results. For example, for the outcome postextubation stridor, the individual-average estimate showed a significant harmful effect (odds ratio [OR] = 1.65, 95% CI = 1.02-2.67), unlike the cluster-average (OR = 1.38, 95% CI = 0.87-2.19) or GEEs(exch) estimate (OR = 1.57, 95% CI = 0.98-2.50).

Informative cluster size can occur in CRTs, and the use of estimators that are not clearly aligned to the target estimand can affect the interpretation of some results.