Incomparability of treatment groups is often blindly ignored in randomised controlled trials - a post hoc analysis of...

Randomisation is often believed to lead to baseline comparability of treatment groups in controlled trials. This study aims to challenge this popular belief, which is relevant in expectation- but not necessarily in realisation.

After presenting an overview of methods for assessing baseline comparability of treatment groups in randomised controlled trials (RCTs), we reviewed RCTs published over 1 year in three high-impact medical journals. We extracted data regarding the methods used to evaluate baseline comparability. To quantify baseline balance, we calculated post hoc standardised mean differences (SMDs) in baseline characteristics reported in these trials.

Amongst 142 RCTs, 120 (84.5%) claimed that baseline comparability was achieved. However, 81 RCTs (57%) did not report how they assessed this balance. The rest (61 RCTs, 43%) used traditional statistical tests, which are deemed inappropriate for balance checking. Our post hoc calculation of SMDs showed that 49 (34.5%) RCTs had at least one baseline variable, which might have been strongly unbalanced (i.e., SMD ≥25%) across treatment groups.

Baseline incomparability of treatment groups in RCTs is often blindly ignored. We suggest it be thoroughly evaluated and transparently reported, using the standardised mean difference or other proper balance metrics.