Controlling for selection bias in the evaluation of Alcoholics Anonymous as aftercare treatment

The purpose of this research was to control for self-selection bias in the evaluation of Alcoholics Anonymous (AA) as aftercare treatment. Observational studies of alcoholism aftercare treatment are subject to selection bias whenever the self-selection process results in important differences in unobserved casemix dimensions between treatment groups.

The sample included 118 male veterans discharged from inpatient alcohol treatment, 85% of whom were followed-up at 3 months. Drinking outcomes were measured by self-reported abstinence in the third month after discharge. The aftercare treatment effect of AA was estimated using standard logistic regression analysis and instrumental variables analysis. Instruments included the subject's ability to drive oneself to AA meetings and the presence/absence of an AA meeting in the subject's town of residence. A Hausman exogeniety test was used to determine whether the standard logistic regression results were subject to self-selection bias.

Estimates from the standard logistic regression yielded a positive (OR = 3.7) and significant (p = .018) treatment effect for AA attendance. However, the instrumental variables analysis yielded a smaller (OR* = 1.7) and insignificant treatment effect estimate (p = .782). The Hausman exogeniety test indicated that the treatment effect estimate from the standard logistic regression was subject to significant self-selection bias (chi2 = 83.9, 1 df, p <.01).

The AA aftercare treatment effect observed in this sample was due to differences in unobserved casemix factors between the treatment groups. Results suggest that previous AA aftercare research may have also been subject to self-selection bias. Researchers of substance abuse outcomes should consider analyzing nonexperimental data using instrumental variables methodologies.