Measuring treatment differentiation for implementation research: the Therapy Process Observational Coding System for...

Observational measures to assess implementation integrity (the extent to which components of an evidence-based treatment are delivered as intended) are needed. The authors evaluated the reliability of the scores and the validity of the score interpretations for the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale (TPOCS-RS; McLeod, 2010) and assessed the potential of the TPOCS-RS to assess treatment differentiation, a component of implementation integrity. The TPOCS-RS includes 5 theory-based subscales (Cognitive, Behavioral, Psychodynamic, Client-Centered, and Family). Using the TPOCS-RS, coders independently rated 954 sessions conducted with 89 children (M age = 10.56, SD = 2.00; age 7-15 years; 65.20% White) diagnosed with a primary anxiety disorder who received different treatments (manual-based vs. nonmanualized) across settings (research vs. practice). Coders produced reliable ratings at the item level (M intraclass correlation coefficient = .76, SD = .18). Analyses support the construct validity of the Cognitive and Behavioral subscale scores and, to a lesser extent, the Psychodynamic, Family, and Client-Centered subscale scores. Correlations among the TPOCS-RS subscale scores and between the TPOCS-RS subscale scores and observational ratings of the alliance and client involvement were moderate suggesting independence of the subscale scores. Moreover, the TPOCS-RS showed promise for assessing implementation integrity as the TPOCS-RS subscale scores, as hypothesized, discriminated between manual-guided treatment delivered across research and practice settings and nonmanualized usual care. The findings support the potential of the TPOCS-RS Cognitive and Behavioral subscales to assess treatment differentiation in implementation research. Results for the remaining subscales are promising, although further research is needed.