The impact of non-blinding in critical care medicine trials

It is uncertain whether, in critical care medicine, non-blinded trials are associated with a bias toward a different effect size. The aim of our study was to assess if conducting non-blinded/open label studies leads to greater effect size than blinded studies, and to provide an estimate of the weight of this difference.

We systematically searched all papers published in peer-reviewed journals between January 2000 and December 2015, dealing with non surgical interventions in critically ill adults and reporting a statistically significant difference in mortality. We assessed the number needed to treat (NNT) of each trial to estimate the treatment effect size and we divided studies into non-blinded, single-blinded and double-blinded. We searched for correlation between the treatment effect size and blinding, and adjusted for the other trial variables.

We identified 119 critically ill randomized controlled trials. Of these, 69 studies were non-blinded and 50 were blinded. The median NNT in non-blinded studies was 5 [IQR 4-7] while it was 7 [IQR 5-7] in the blinded studies (p < .001).

The NNT for blinded studies is 40% higher than for unblinded studies. This should be taken into account when planning and interpreting the findings of non-blinded studies performed in critically ill settings.