Related Experiment Video
Updated: Jul 17, 2025

A Clinical Trial Assessing the Safety, Efficacy, and Delivery of Olive-Oil-Based Three-Chamber Bags for Parenteral Nutrition
Published on: September 20, 2019
Treatment Effect Estimates From Pilot Trials Are Unreliable.
Jesse D Troy1, Megan L Neely1, Gina-Maria Pomann1
1Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, North Carolina, USA.
Pilot trials are unreliable for assessing treatment efficacy due to sampling variability. Computer simulations show that these small studies often yield misleading results, incorrectly suggesting harm or benefit, and should not guide treatment development decisions.
Area of Science:
- Clinical Trials Methodology
- Biostatistics
- Translational Research
Background:
- Pilot trials are small-scale studies to inform larger efficacy trials.
- The CONSORT guideline defines pilot trials for feasibility, not efficacy assessment.
- Sampling variability poses a significant challenge to the reliability of pilot trial efficacy estimates.
Purpose of the Study:
- To demonstrate the influence of sampling variability on efficacy estimates from pilot trials using computer simulation.
- To illustrate why pilot trial designs are inappropriate for evaluating treatment promise.
Main Methods:
- Computer simulation of a 2-arm parallel group trial with a survival outcome (N=20 per group).
- Simulations conducted under two scenarios: treatment efficacious (hazard ratio [HR]=0.75) and non-efficacious (HR=1).
Main Results:
- In simulations where treatment was efficacious (HR=0.75), ~20% of trials incorrectly suggested harm (HR>1).
- In simulations where treatment was not efficacious (HR=1), 50% of trials incorrectly suggested benefit.
- Observed results in simulations were distributed around the true treatment effect.
Conclusions:
- Treatment effect estimates from pilot trials are highly prone to misleading conclusions.
- Pilot trials should not be used to make future development decisions for novel therapies due to high risk of erroneous findings.
- The statistical phenomenon of sampling variability undermines the reliability of pilot trials for efficacy assessment.
More Related Videos
07:02A Computerized Test Battery to Study Pharmacodynamic Effects on the Central Nervous System of Cholinergic Drugs in Early Phase Drug Development
Published on: February 11, 2019
07:31A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills
Published on: February 13, 2020
Related Concept Videos
Regression Toward the Mean
Blind Procedures
Clinical Trials
There are four phases in a clinical trial. A phase one...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
The Placebo Effect
Blinding