Multilevel Matching in Natural Experimental Studies: Application to Stepping up Counties

Among many approaches for selecting match control cases, few methods exist for natural experiments (Li, Zaslavsky & Landrum, 2007), especially when studying clustered or hierarchical data. The lack of randomization of treatment exposure gives importance to using proper statistical procedures that control for individual differences. In this natural experimental study, which has a hierarchical structure, we plan to evaluate the efforts of 455 counties across the United States to make targeted efforts to improve mental health services and reduce jail utilization over time. Nested within states, counties are clustered on health and social indicators, which affect the likelihood of making improvements in these areas. Similar to a randomized trial, prior to collecting survey data, it is necessary to identify matched control counties as study sites based on an array of state and county covariates. Accounting for the hierarchal structure of data, a blend of various probability-based models are presented to achieve this goal. Methods include multivariable models that control for observed differences among treatment and control groups, shrinkage based LASSO as a variable selection technique, and logistic models.