Developing a systematic evaluation approach for training programs within a train-the-trainer model for youth...

The purpose of this small pilot study was three-fold: (a) to begin development of a coding scheme for supervisor and therapist skill acquisition, (b) to preliminarily investigate a pilot train-the-trainer paradigm for skill development, and (c) to evaluate self-reported versus observed indicators of skill mastery in that pilot program. Participants included four supervisor-therapist dyads (N = 8) working with public mental health sector youth. Master trainers taught cognitive-behavioral therapy techniques to supervisors, who in turn trained therapists on these techniques. Supervisor and therapist skill acquisition and supervisor use of teaching strategies were repeatedly assessed through coding of scripted role-plays with a multiple-baseline across participants and behaviors design. The coding system, the Practice Element Train the Trainer - Supervisor/Therapist Versions of the Therapy Process Observational Coding System for Child Psychotherapy, was developed and evaluated though the course of the investigation. The coding scheme demonstrated excellent reliability (ICCs [1,2] = 0.81-0.91) across 168 video recordings. As calculated through within-subject effect sizes, supervisor and therapist participants, respectively, evidenced skill improvements related to teaching and performing therapy techniques. Self-reported indicators of skill mastery were inflated in comparison to observed skill mastery. Findings lend initial support for further developing an evaluative approach for a train-the-trainer effort focused on disseminating evidence-based practices.