Approaches to risk-adjusting outcome measures applied to criminal justice involvement after community service

The ethic of fairness in program evaluation requires that measures of behavioral health agency performance be sensitive to differences in those agencies' caseload composition. The authors describe two traditional approaches to the statistical risk adjustment of outcome measures (stratification weighting and pre-post measurement) that are designed to account for differences in caseload composition and introduce a method that incorporates the strengths of both approaches. Procedures for deriving each of these measures are described in detail and demonstrated in the evaluation of a statewide system of community-based behavioral health care programs. This evaluation examines the degree to which service recipients get into trouble with the law after treatment. Three measures are recommended for inclusion in outcome-oriented "report cards," and the interpretation of each measure is discussed. Finally, the authors suggest formats for graphic and tabular presentation of the risk-adjusted evaluation for sharing findings with diverse stakeholder groups.