Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials

Area of Science:

• Health Services Research
• Biostatistics
• Pharmacoepidemiology

Background:

• Insurance claims databases offer valuable real-world evidence (RWE) on medical product effectiveness.
• Concerns persist regarding potential bias in RWE from nonrandomized studies due to lack of baseline randomization and measurement issues.

Purpose of the Study:

• To emulate the designs of 32 randomized clinical trials (RCTs) using observational database studies.
• To quantify the agreement between RCT results and their emulated database counterparts.

Main Methods:

• Conducted new-user cohort studies with propensity score matching across three US claims databases.
• Emulated RCT design parameters (population, intervention, comparator, outcome, time [PICOT]) for 30 completed and 2 ongoing RCTs.
• Registered all 32 protocols on ClinicalTrials.gov prior to analysis.

Main Results:

• Overall agreement between RCTs and database emulations showed a Pearson correlation of 0.82.
• Concordance was higher (Pearson r = 0.93) when RCT design and measurements were closely emulated.
• Weaker concordance (Pearson r = 0.53) was observed when close emulation of PICOT elements was not feasible with claims data.

Conclusions:

• RWE studies can align with RCT conclusions if designs and measurements are closely emulated.
• Achieving precise emulation with claims data can be challenging, leading to varied concordance.
• Divergence in results may stem from emulation differences, chance, or residual confounding, which are difficult to isolate.