Evaluating the Quality of Rural Intensive Case Management Services using Administrative Data: an Exploratory Study

Area of Science:

• Health Services Research
• Mental Health Services
• Rural Health

Background:

• Assertive Community Treatment (ACT) is effective for severe mental illness (SMI) and is expanding in rural settings.
• Evaluating ACT fidelity in rural areas requires cost-effective methods to ensure intended service delivery.
• The Veterans Health Administration (VHA) implemented an ACT-like program (Mental Health Intensive Case Management - ACT/ICM) for rural SMI veterans.

Purpose of the Study:

• To demonstrate the use of administrative data for characterizing services in rural ACT/ICM programs.
• To compare rural ACT/ICM programs with general mental health programs and with each other.
• To identify programs conforming to a typical rural ACT/ICM model using low-cost methods.

Main Methods:

• Utilized administrative data from 298,509 veterans receiving mental health services in FY 2012.
• Employed logistic regression to identify characteristics distinguishing ACT/ICM patients.
• Applied propensity scores to identify programs most and least conforming to the ACT/ICM profile.

Main Results:

• ACT/ICM patients were characterized by diagnoses of schizophrenia, bipolar disorder, major depression, and high service utilization.
• The logistic regression model achieved a high c statistic of 0.91.
• Propensity scores successfully differentiated programs based on fidelity to the rural ACT/ICM model.

Conclusions:

• Low-cost administrative data can effectively identify programs adhering to an empirically derived rural ACT/ICM model.
• This approach supports quality assurance for ACT services in rural mental health settings.
• Further validation of this administrative data-driven fidelity evaluation is recommended.