Related Experiment Videos
'Optimism bias' in contemporary national clinical trial network phase III trials: are we improving?
Kaveh Zakeri1, Sonal Noticewala2, Lucas Vitzthum2
1Radiation Research Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, Rockville, USA; Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, USA.
Background:
Previous studies have found that overestimating treatment effects (i.e. 'optimism bias') leads to underpowered clinical trials. The prevalence of 'optimism bias' in contemporary National Clinical Trials Network (NCTN) cancer clinical trials is unknown.
Methods:
We conducted a systematic review of NCTN phase III randomized trials published from January 2007 to January 2017. We compared the hypothesized versus observed treatment effects in each trial, and examined whether trial-related factors were correlated with the study results. We also reviewed the methods of each protocol to assess whether a rationale for the hypothesized effect size was provided.
Results:
We identified 161 clinical trials, of which 130 were eligible for analysis. Original protocols could not be located for 8 trials (5.0%). Twenty-eight trials (21.5%) observed a statistically significant difference in the primary end point favoring the experimental treatment. The median ratio of observed-to-expected hazard ratios among trials that observed a statistically significant effect on the primary end point was 1.07 (range: 0.33-1.28) versus 1.32 (range: 0.86-2.02) for trials that did not, compared with 1.34 and 1.86, respectively, for National Cancer Institute (NCI) trials published between 1955 and 2006. An effect size at least as large as the one projected in the protocol trials was observed in 9.8% of trials, compared with 17% of NCI trials published from 1955 to 2006. Most trials (64.6%) provided no rationale to support the magnitude of the proposed treatment effect in the protocol.
Conclusions:
Despite a reduction in 'optimism bias' compared with previous eras, most contemporary NCTN phase III trials failed to establish statistically significant benefits of new cancer therapies. Better rationalization of proposed effect sizes in research protocols is needed.
Insights
Optimism bias in cancer trials is reduced but still present. Most National Clinical Trials Network (NCTN) trials do not show significant treatment benefits, highlighting the need for better effect size rationalization in protocols.
Area of Science:
- Oncology
- Clinical Trials
- Biostatistics
Background:
- Overestimating treatment effects, known as optimism bias, has been linked to underpowered clinical trials.
- The prevalence of optimism bias in contemporary National Clinical Trials Network (NCTN) cancer clinical trials remains largely unknown.
Purpose of the Study:
- To determine the prevalence of optimism bias in NCTN phase III cancer clinical trials.
- To assess the correlation between trial-related factors and study outcomes.
- To evaluate the provision of rationales for hypothesized effect sizes in trial protocols.
Main Methods:
- Systematic review of NCTN phase III randomized trials published between January 2007 and January 2017.
- Comparison of hypothesized versus observed treatment effects.
- Review of trial protocols for rationales supporting effect size estimations.
Main Results:
- Of 130 eligible trials, 21.5% showed statistically significant benefits for experimental treatments.
- The median ratio of observed-to-expected hazard ratios was 1.07 for significant trials, compared to 1.32 for non-significant trials.
- Only 9.8% of trials observed an effect size as large as projected, and 64.6% lacked a rationale for the proposed effect size.
Conclusions:
- Contemporary NCTN trials exhibit reduced optimism bias compared to earlier periods.
- However, most NCTN phase III trials still fail to demonstrate statistically significant benefits for new cancer therapies.
- Improved rationalization of proposed effect sizes in research protocols is crucial for future trial design and success.