Risk adjustment of heroin treatment outcomes for comparative performance assessment in England

Variability in effectiveness of treatment for substance abuse disorder (SUD) is an important and understudied issue. This study aimed to quantify the extent of outcome variability in the English SUD treatment system after adjusting for potential confounding variables.

Prospective cohort study using data from the English national drug treatment outcome monitoring database.

All 149 administrative areas delivering publicly funded SUD services in the National Health Service and non-governmental sector.

New adult admissions between January 2008 and October 2010 with illicit heroin-related problems in all administrative areas, with an in-treatment review conducted between 5 and 26 weeks (mean = 129.5 days; SD = 40.0) up to 30 April 2011 (n = 65 223; 75.6% of eligible clients). Individuals were divided randomly to form model developmental and internal validation samples. These were contrasted with an independent (external) sample of the same population admitted to treatment between November 2010 and April 2011 and followed to 31 October 2011 (n = 13 797; 81.4% of those eligible).

The outcome measure was self-reported illicit heroin use, categorized as abstinent or deteriorated (the latter by Reliable Change Index), each risk-adjusted by person-level (demographics, clinical severity and treatment complexity) and area-level (SUD prevalence, social deprivation and severity averages) covariates by multivariable logistic regression using multiply imputed outcome and covariate data. Risk-adjusted models were assessed by information criteria and discrimination (c-index). Standardized outcome rates were compared by funnel plot with 95% and 99% control limits.

Models of heroin abstinence (48.4%) and deterioration (3.2%) were comparable across the developmental and validation samples (c-index = 0.70-0.71 and 0.82-0.87), with 79.2 and 94.0%, respectively, of the 149 treatment areas falling within 95% control limits. At the 99% limit, seven areas (4.7%) achieved abstinence rates above the national average, and eight had relatively poor abstinence rates (5.4%). At the 99% control limit, one area achieved very low deterioration outcomes and two (1.3%) were worse that the average. Risk adjustment served to increase abstinence rates in good performing areas by 0.63% and reduce abstinence rates by 0.37% in poor performing areas, and by 0.12% and 0.18%, respectively, for deterioration.

There is some exceptional variability in the apparent effectiveness of the English treatment system for substance use disorders. It is important to determine the source of this variability in order to inform drug treatment delivery and its evaluation both in England and overseas.