Reliability of adherence and competence assessment in psychoeducational treatment: influence of clinical experience

For the evaluation of therapist competence in psychotherapeutic treatment, only highly experienced judges (experts) were found to be adequate, whereas therapist adherence could be assessed by nonexperts. Using experts implies high costs for the rating process. Therefore, an interesting question is whether experts are also necessary for the assessment of therapist adherence and competence in psychoeducational treatment. To test this, four judges evaluated therapist adherence and competence in 30 randomly selected videotaped sessions of manualized psychoeducation for recurrent depression. Two judges exhibited high clinical experience (experts) while two judges did not (novices). We could demonstrate that the novices were also able to evaluate therapists' adherence and competence in psychoeducative treatment with high reliability. Moreover, expert judgments were not more reliable than novice judgments. Adherence and competence ratings of experts and novices showed high concordance. These results carry implications in terms of reducing costs associated with the judgment process.