Related Experiment Video
Updated: Dec 6, 2025
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Powering Bias and Clinically Important Treatment Effects in Randomized Trials of Critical Illness
Darryl Abrams1, Sydney B Montesi2, Sarah K L Moore3
1Center for Acute Respiratory Failure and Division of Pulmonary, Allergy, and Critical Care Medicine, Columbia University College of Physicians and Surgeons and New York-Presbyterian Hospital, New York, NY.
Objectives:
Recurring issues in clinical trial design may bias results toward the null, yielding findings inconclusive for treatment effects. This study evaluated for powering bias among high-impact critical care trials and the associated risk of masking clinically important treatment effects.
Design, Setting, And Patients:
Secondary analysis of multicenter randomized trials of critically ill adults in which mortality was the main endpoint. Trials were eligible for inclusion if published between 2008 and 2018 in leading journals. Analyses evaluated for accuracy of estimated control group mortality, adaptive sample size strategy, plausibility of predicted treatment effect, and results relative to the minimal clinically important difference. The main outcome was the mortality risk difference at the study-specific follow-up interval.
Interventions:
None.
Measurements And Main Results:
Of 101 included trials, 12 met statistical significance for their main endpoint, five for increased intervention-associated mortality. Most trials (77.3%) overestimated control group mortality in power calculations (observed minus predicted difference, -6.7% ± 9.8%; p < 0.01). Due to this misestimation of control group mortality, in 14 trials, the intervention would have had to prevent at least half of all deaths to achieve the hypothesized treatment effect. Seven trials prespecified adaptive sample size strategies that might have mitigated this issue. The observed risk difference for mortality fell within 5% of predicted in 20 trials, of which 16 did not reach statistical significance. Half of trials (47.0%) were powered for an absolute risk reduction greater than or equal to 10%, but this effect size was observed in only three trials with a statistically significant treatment benefit. Most trials (67.3%) could not exclude clinically important treatment benefit or harm.
Conclusions:
The design of most high-impact critical care trials biased results toward the null by overestimating control group mortality and powering for unrealistic treatment effects. Clinically important treatment effects often cannot be excluded.
More Related Videos
04:53A Clinical Trial Assessing the Safety, Efficacy, and Delivery of Olive-Oil-Based Three-Chamber Bags for Parenteral Nutrition
Published on: September 20, 2019
05:16Cutoff Value of Phase Angle by Bioelectrical Impedance Analysis at Admission as a Prognostic Factor in Patients with Acute Heart Failure
Published on: June 10, 2025
Related Concept Videos
01:11Blinding
01:29Bias in Epidemiological Studies
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
02:07Blind Procedures
01:16Clinical Trials
There are four phases in a clinical trial. A phase one...
01:22Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...