Reliability of multidimensional substance abuse treatment matching: implementing the ASAM Patient Placement Criteria

For meaningful adoption, the Patient Placement Criteria (PPC) of the American Society of Addiction Medicine (ASAM) will need adequate interrater reliability. In a decision analysis of the original PPC, we reduced potential sources of unreliability, mapped question items from clinical research instruments to each decision point, and programmed the item map as a computerized structured interview. Then, target videotapes from eight substance dependent adults who had been distributed by the algorithm into three levels of care (LOC) were independently scored by four raters who were blind as to ASAM LOC. The intraclass correlation coefficient for ASAM LOC assignment was .77. For all but two subscales of component instruments, values were above .70 and significant, indicating high interrater reliability. With these methods, excellent reliability is possible for complex decision trees, making it possible to improve the validity of the ASAM Criteria and similar complex hierarchical clinical protocols.