Identifiability and selection bias in indirect comparisons using real-world controls: a systematic review of methods...

Recently, the integration of external controls to construct an externally controlled trial has gained attraction as a way to strengthen the level of evidence of single-arm trials (SATs). We aimed to systematically evaluate the reporting of key methodological features of indirect comparisons that use real-world (RW) controls to supplement single-arm drug trials.

We conducted a systematic review to critically appraise the methodological features of these designs and identify key omissions. We compared studies based on methods that address selection bias by targeting covariate balance (eg, matching, weighting, g-computation, or doubly robust estimators) ("balancing-based methods"), with those which do not explicitly target imbalances, using outcome regression or unadjusted analyses for group adjustment ("other methods").

Our search identified 55 single-arm drug trials published from 1987 to 2024. Of these, 29 were described as using balancing-based methods and 26 used other methods. Over half of the trials were in oncology, with hematologic-oncology being the most represented pathology. Only 29% of studies provided a rationale for using an indirect comparison of the SAT to real-world data (RWD). RW control patients were recruited over longer and earlier time periods than patients in the SAT, compromising positivity assumption. Balancing-based methods were more recent compared to other methods (median of publication time, 2023 vs 2019), with the indirect comparison often following a previous publication of the trial (66% vs 8%). In the balancing-based group, sample sizes in both the experimental (median 77 vs 31) and control (median 97 vs 40) groups were higher, as well as the number of participating centers (with monocentric studies in 30% vs 57%). Assessment of postadjustment balance was reported more frequently in the balancing-based group, though often only based on P values (34% vs 62%), improved for those using weighting compared to matching approaches.

To ensure the validity of unanchored indirect comparisons with RWD controls, future studies should adopt and transparently report on robust methodological frameworks based on the principles of causal inference.

Some drug trials do not include a comparison group ("single-arm trials [SATs]"), making it difficult to judge how well a treatment works in comparison to the standard of care (SoC) at the time. An increasingly popular solution is to compare patients in the SAT with similar patients identified from data collected outside of the clinical trial setting (such as medical records, administrative claims, and registries), also called "real-world data" (RWD). However, these observational data are of lower quality, lacks important information. Moreover, an indirect comparison of these two data sources (single-arm and real-world [RW]) requires adequate statistical analysis methods to avoid biased results. Our goal was to evaluate the clarity of reporting and the rigor of the analyses in these published comparisons. We evaluated a total of 55 studies published from 1987 to 2024. Most studies were in the field of oncology. Fewer than one-third of the studies explained why they chose to use RWD instead of trial data. RW control patients were recruited over longer and earlier time periods than patients in the SAT. The sample sizes in both the treated and control groups were larger, the groups were more balanced when the studies used recent statistical methods to control for systematic differences between them. Overall, our findings suggest that these studies do not report all the elements necessary to conclude that their findings are clinically relevant or reproducible.