Strengthening Real-World Evidence: Recognizing and Overcoming Methodological Pitfalls in Observational Health Data

Real-world health data (RWD), typically derived from electronic health records, administrative claims, patient registries, wearable devices, and other non-experimental sources, are increasingly used to generate real-world evidence (RWE) in public health and clinical medicine. While RWD offers multiple advantages over traditional randomized clinical trials (RCTs) in cost, feasibility, speed, and generalizability, methodological pitfalls can undermine the reproducibility and validity of findings based on RWD analysis.

In this manuscript, we highlight key methodological challenges in RWD analysis and illustrate them with published case studies. We then review published methodological solutions, including target trial emulation, propensity score adjustment, instrumental variable approaches, and inverse probability of sampling weighting, that can improve the validity and generalizability of empirical results derived from RWD.

We contend that awareness of RWD limitations is not a justification for avoiding its use, but a mandate to use it more carefully.

Pre-specifying sampling and analytic plans, applying robust inferential methods, and continuing to invest in novel bias-reduction methods are essential to ensuring that RWE fulfills its promise for public health research and policy.