The design of the MBT-G adherence and quality scale

Few group psychotherapy studies focus on therapists' interventions, and instruments that can measure group psychotherapy treatment fidelity are scarce. The aim of the present study was to evaluate the reliability of the Mentalization-based Group Therapy Adherence and Quality Scale (MBT-G-AQS), which is a 19-item scale developed to measure adherence and quality in mentalization-based group therapy (MBT-G). Eight MBT groups and eight psychodynamic groups (a total of 16 videotaped therapy sessions) were rated independently by five raters. All groups were long-term, outpatient psychotherapy groups with 1.5 hours weekly sessions. Data were analysed by a Generalizability Study (G-study and D-study). The generalizability models included analyses of reliability for different numbers of raters. The global (overall) ratings for adherence and quality showed high to excellent reliability for all numbers of raters (the reliability by use of five raters was 0.97 for adherence and 0.96 for quality). The mean reliability for all 19 items for a single rater was 0.57 (item range 0.26-0.86) for adherence, and 0.62 (item range 0.26-0.83) for quality. The reliability for two raters obtained mean absolute G-coefficients on 0.71 (item range 0.41-0.92 for the different items) for adherence and 0.76 (item range 0.42-0.91) for quality. With all five raters the mean absolute G-coefficient for adherence was 0.86 (item range 0.63-0.97) and 0.88 for quality (item range 0.64-0.96). The study demonstrates high reliability of ratings of MBT-G-AQS. In models differentiating between different numbers of raters, reliability was particularly high when including several raters, but was also acceptable for two raters. For practical purposes, the MBT-G-AQS can be used for training, supervision and psychotherapy research.