Implications of the Propensity Score Matching Paradox in Pharmacoepidemiology

Recent work has demonstrated that propensity score matching may lead to increased covariate imbalance, even with the corresponding decrease in propensity score distance between matched units. The extent to which this paradoxical phenomenon might harm causal inference in real epidemiologic studies has not been explored. We evaluated the effect of this phenomenon using insurance claims data from the Pharmaceutical Assistance Contract for the Elderly (1999-2002) and Medicaid Analytic eXtract (2000-2007) databases in the United States. For each data set, we created several 1:1 propensity-score-matched data sets by manipulating the size of the covariate set used to generate propensity scores, the index exposure prevalence in the prematched data set, and the matching algorithm. We matched all index units, then progressively pruned matched sets in order of decreasing propensity score distance, calculating covariate imbalance after each pruning. Although covariate imbalance sometimes increased after progressive pruning of matched sets, the application of commonly used propensity score calipers for defining an acceptable match stopped pruning near the lowest region of the imbalance trend and resulted in an improvement over the imbalance in the prematched data set. Thus, propensity score matching does not appear to induce increased covariate imbalance when standard propensity score calipers are applied in these types of pharmacoepidemiologic studies.