Evaluating and improving a matched comparison of antidepressants and bone density

Matching is a common approach to covariate adjustment in estimating causal effects in observational studies. It is important to assess covariate balance of the matched samples. This is usually done informally, in ways that have a number of limitations. First, there are many diagnostics, even if covariates are assessed one at a time, which raises multiplicity issues. In addition, joint distributions of covariates, even bivariate distributions, are often ignored. Further, it is an open question whether diagnostics identify the major problems. To address these issues, a formal assessment of covariate balance is developed in the current paper. Unlike the common informal diagnostics, the proposed method compares both marginal distributions and joint distributions of the matched sample with those of the benchmark, complete randomizations. The method controls the probability of falsely identifying a covariate imbalance among many comparisons, yet it has a high probability of correctly detecting and identifying a major problem. An R package met implementing the proposed method is available on CRAN.