Evaluating large-scale propensity score performance through real-world and synthetic data experiments.

Area of Science:

• Epidemiology
• Biostatistics
• Health Services Research

Background:

• Propensity score adjustment is crucial for controlling confounding in observational studies.
• Developing robust frameworks to assess propensity score performance and optimize model selection is essential for large-scale research.
• Existing methods require rigorous evaluation to ensure reliable confounding control.

Purpose of the Study:

• To introduce and evaluate a comprehensive framework for propensity score performance assessment.
• To compare the effectiveness of L1-regularized regression and high-dimensional propensity score (hdPS) methods in confounding control.
• To identify optimal propensity score model selection strategies for observational data.

Main Methods:

• Developed a propensity score evaluation framework using synthetic (plasmode) and real-world data.
• Simulated survival data with known effect sizes for synthetic experiments.
• Compared L1-regularized regression and hdPS using negative control outcomes and evaluated performance metrics.

Main Results:

• L1-regularization demonstrated superior model fit, covariate balance, and negative control bias reduction over hdPS.
• Simulation results under proportional hazards were variable and dependent on specific parameters.
• Incorporating regularization into hdPS addressed non-convergence but minimally impacted overall performance.

Conclusions:

• L1-regularization, by incorporating all covariates simultaneously, provides superior propensity score performance compared to the hdPS marginal screen.
• The proposed evaluation framework offers a reliable method for assessing propensity score methods.
• Findings highlight the advantages of simultaneous covariate inclusion for robust confounding control.