Single-arm interventional versus observational studies for assessing efficacy: A meta-epidemiological study

Interventional single-arm trials (SATs) are increasingly being used as evidence, despite a lack of agreement on their validity and where they should sit in the hierarchy of evidence. We conducted a meta-epidemiological study to investigate whether there are systematic differences in outcomes and levels of between-study heterogeneity for SATs compared with their observational counterpart, single-arm cohort studies.

We identified systematic reviews (SRs) of pharmacological interventions, published in 2023, that included both interventional and observational single-arm studies. For each SR, subgroup meta-analysis of dichotomous outcomes was conducted for included SATs and single-arm cohort studies to assess effect sizes, levels of heterogeneity and between group differences. In a sensitivity analysis, clinically heterogeneous primary studies were removed and analyses re-run.

66 SRs contained single-arm studies, of which 13 reported meta-analyses of dichotomous efficacy outcomes. There was no overall risk difference for SATs compared with single-arm cohort studies (risk difference: -0.020, 95% CI: -0.092 to 0.052, p = 0.59). In the sensitivity analysis, there was a tendency to higher effect for single-arm cohort studies, but no significant difference (risk difference: -0.071, 95% CI: -0.161, 0.019, p = 0.12). There were high levels of between-study heterogeneity within both SATs (median; range I ²: 54.8; 11.3-91.0) and single-arm cohorts (median; range I ²: 77.2; 0-94.7) and heterogeneity remained high in the sensitivity analysis.

There do not appear to be systematic differences in outcome between SATs and single-arm cohort studies, but further research is recommended to confirm this finding. Levels of heterogeneity are high within both designs, even after attempts to reduce clinical heterogeneity. Because clinical heterogeneity had potentially been removed, remaining statistical heterogeneity may have been due to bias related to study conduct. Future work should utilize larger samples and additional methods to further clarify the relative validity of single-arm designs.