Internal Validity-Gated Dual Weighting: Toward Target Validity in the Synthesis of Randomized Controlled Trials and...

Integrating randomized controlled trials (RCTs) and real-world studies (RWS) requires balancing internal and external validity to achieve target validity. We extended external validity assessment to better capture contextual complexity and diversity, integrating it with established internal validity tools to refine the target validity scoring system. Then, we proposed an Internal Validity-Gated Dual Weighting (IVDW) strategy, which applies external validity weights to studies exceeding a predefined internal validity threshold.

The scoring system was refined through a two-round Delphi process and Analytic Hierarchy Process to ensure content validity and derive item weights. IVDW was implemented in both frequentist and Bayesian frameworks and compared with four alternatives: Equal-Weighting, Dual-dimensional Weighting (DDW), Conditional Exclusion-based Dual-weighting (CEDW), and Internal-validity-Only Weighting (IVOW), in a novel synthesis of neoadjuvant chemoimmunotherapy in older adults with non-small cell lung cancer.

Studies with concurrently high internal and external validity had risk difference estimates below 0.2. IVDW more effectively downweighted low-internal-validity studies than DDW and reduced reliance on low-external-validity studies compared to IVOW. It improved interpretability by explicitly balancing validity dimensions while maintaining higher evidence inclusion than CEDW. When combined with the quality effects model, IVDW produced estimates aligned with high target validity studies.

The proposed target validity scoring system, combined with the IVDW strategy, steers effect estimates toward studies with higher internal and external validity. It extends a novel perspective for evidence synthesis and enhances the external validity of meta-analytic findings in real-world clinical decision-making.