Patient-rated alliance as a measure of therapist performance in two clinical settings

The ability to form a strong therapeutic alliance is considered a foundational skill across psychotherapies. Patient-rated measures of the alliance are now being used to make judgments about a therapist's tendency to build alliances with their patients. However, whether a patient-rated alliance measure provides a useful index of a therapist's tendency to have strong alliances is not clear.

We examined therapist differences in patient ratings of the alliance obtained from an HMO that included 2 samples--an internal HMO clinic (n = 3287, 3781 ratings, therapist n = 72) and an independent practice (IP) based sample (n = 1320, 1690 ratings, therapist n = 93). First, we estimated the amount of variability in alliance scores due to therapist, including the consistency of estimated differences across 2 samples and using a cross-validation strategy. Second, we used a multivariate multilevel model to examine the convergent and discriminant validity of therapist differences in patient alliance ratings in items theoretically related (e.g., satisfaction) and unrelated (e.g., patient ratings of scheduling staff) to the therapist and alliance.

Therapists accounted for between 1.74% and 6.93% of the variability in alliance ratings, with greater differences among IP therapists. Therapist differences were generally stable and unaffected by case mix. In addition, therapist differences in alliance were correlated with therapist differences in similar items but were relatively unrelated to theoretically distinct items.

Therapist differences in the alliance were small but, with a sufficient number of ratings, may provide reliable information regarding a therapist's tendency to form strong alliances with their patients.