Per-Protocol analyses produced larger treatment effect sizes than intention to treat: a meta-epidemiological study

To undertake meta-analysis and compare treatment effects estimated by the intention-to-treat (ITT) method and per-protocol (PP) method in randomized controlled trials (RCTs). PP excludes trial participants who are non-adherent to trial protocol in terms of eligibility, interventions, or outcome assessment.

Five high impact journals were searched for all RCTs published between July 2017 to June 2019. Primary outcome was a pooled estimate that quantified the difference between the treatment effects estimated by the two methods. Results are presented as ratio of odds ratios (ROR). Meta-regression was used to explore the association between level of trial protocol non-adherence and treatment effect. Sensitivity analyses compared results with varying within-study correlations and across various study characteristics.

Random-effects meta-analysis (N = 156) showed that PP estimates were on average 2% greater compared to the ITT estimates (ROR: 1.02, 95% CI: 1.00-1.04, P = 0.03). The divergence further increased with higher degree of protocol non-adherence. Sensitivity analyses reassured consistent results with various within-study correlations and across various study characteristics.

There was evidence of larger treatment effect with PP compared to ITT analysis. PP analysis should not be used to assess the impact of protocol non-adherence in RCTs. Instead, in addition to ITT, investigators should consider randomization based casual method such as Complier Average Causal Effect (CACE).