Estimating treatment effects from a randomized controlled trial with mid-trial design changes

Area of Science:

• Clinical Trials Methodology
• Biostatistics
• Health Services Research

Background:

• Unplanned design modifications in randomized controlled trials (RCTs) are rarely reported.
• Combining data from pre- and post-design changes can introduce bias and limit interpretability.
• Estimating treatment effects in RCTs with mid-trial design changes requires careful statistical consideration.

Purpose of the Study:

• To examine the statistical implications of major design changes on treatment effect estimates in RCTs.
• To evaluate different statistical approaches for analyzing data from RCTs with mid-trial modifications.
• To provide guidance on appropriate methods for estimating treatment effects when RCT designs change.

Main Methods:

• Utilized a recently completed RCT with two major mid-trial design changes as a case study.
• Conducted a simulation study to mimic design modifications and generate patient-level data.
• Compared statistical properties (bias, MSE, coverage) of naive data lumping, fixed-effect, and random-effect meta-analysis models.

Main Results:

• When between-design heterogeneity was negligible, fixed- and random-effect meta-analysis models provided accurate and precise estimates.
• With increased heterogeneity, random-effect models showed less bias and higher coverage but greater uncertainty (higher MSE) due to fewer studies.
• Increasing within-study sample sizes improved precision and statistical power for effect-size estimates.

Conclusions:

• Naïve data lumping is inappropriate for RCTs with unplanned major design changes.
• Careful selection of statistical approaches, such as meta-analysis, is essential for valid treatment effect estimation.
• Transparency in reporting design changes and their analytical implications is crucial for trial validity.