Evaluating the Quality of Rural Intensive Case Management Services using Administrative Data: an Exploratory Study

Assertive Community Treatment (ACT) for people with severe mental illness is an effective approach that is increasingly implemented in rural areas. Low-cost methods of evaluating fidelity to program models are needed to assure services are delivered as intended. In 2007, the Veterans Health Administration implemented an ACT-like Mental Health Intensive Case Management (ACT/ICM) program for SMI veterans in rural areas. This study demonstrates the use of administrative data, reflecting patient characteristics and intensity of service delivery, to characterize services delivered by these programs, to compare them to general mental health programs at the same VA medical centers, and to each other. A total of 298,509 veterans received mental health services at VA medical centers that also operated a rural ACT/ICM program in FY 2012. Altogether 854 (0.29%) received ACT/ICM services for 1 year or more (long term participants) and 259 (.09%) received them for less than 1 year (new entrants). Logistic regression showed ACT/ICM patients were distinguished by diagnoses of schizophrenia, bipolar disorder, and major depression; larger numbers of psychiatric or substance abuse visits; and use of 3 or more classes of psychotropic medication. The model had a high c statistic of 0.91. Propensity scores allowed clear identification of programs most and least conforming to the profile of a "typical" rural ACT/ICM program. Low cost administrative data can be used to identify programs successfully conforming to an empirically derived rural model of ACT/ICM. Further validation of this approach is needed.