Bias in psychotherapist ratings of client transference and insight

The psychology literature at large considers rater bias to be a substantial source of error in observer ratings. Yet, it is typically ignored by psychotherapy researchers using participant (psychotherapist/client) ratings. In particular, interrater variability, or differences between raters' overall tendency to rate others favorably or unfavorably, has been a largely ignored source of error in studies that use psychotherapists and/or clients as raters. Ignoring rater bias can have serious consequences for statistical power and for interpretation of research findings. Rater bias may be a particular problem in psychotherapy research, as psychotherapists are often asked to rate subjective variables that require much rater inference. Consequently, we examined the extent to which rater bias is a factor in psychotherapist ratings of client transference and insight, by comparing psychotherapist variance from these ratings to psychotherapist variance in ratings of client-perceived emotional intelligence, using Hierarchical Linear Modeling. Results suggest that bias may be a substantial source of error in psychotherapist process and relationship ratings, accounting for, on average, 38% of the total variance in scores, and 30% after accounting for perceived emotional intelligence. (PsycINFO Database Record (c) 2010 APA, all rights reserved).