A comparison of approaches for stratifying on the propensity score to reduce bias

Stratification is a popular propensity score (PS) adjustment technique. It has been shown that stratifying the PS into 5 quantiles can remove over 90% of the bias due to the covariates used to generate the PS. Because of this finding, many investigators partition their data into 5 quantiles of the PS without examining whether a more robust solution (one that increases covariate balance while potentially reducing bias in the outcome analysis) can be found for their data. Two approaches (referred to herein as PSCORE and PSTRATA) obtain the optimal stratification solution by repeatedly dividing the data into strata until balance is achieved between treatment and control groups on the PS. These algorithms differ in how they partition the data, and it is not known which is better, or if either is better than a 5-quantile default approach, for reducing bias in treatment effect estimates.

Monte Carlo simulations and empirical data are used to assess whether PS strata defined by PSCORE, PSTRATA, or 5 quantiles is best at reducing bias in treatment effect estimates, when used within a marginal mean weighting framework (MMWS). These estimates are further compared to results derived using inverse probability of treatment weights (IPTW).

PSTRATA was slightly better than PSCORE in balancing covariates and reducing bias, while both approaches outperformed the 5-quantile approach. Overall MMWS using any stratification method outperformed IPTW.

Investigators should routinely use stratification approaches that obtain the optimal stratification solution, rather than simply partitioning the data into 5 quantiles of the PS. Moreover, MMWS (in conjunction with an optimal stratification approach) should be considered as an alternative to IPTW in studies that use PS weights.