Impact of training on rating competence in cognitive therapy

This naturalistic study examines the impact of a training package on the inter-rater reliability scores of a scale devised to measure competence in cognitive therapy (CT). The study was conducted at the Newcastle Cognitive and Behavioural Therapies Centre, where trainees are routinely required to submit videotapes of their therapeutic work for marking independently by two supervisors. This process was used as vehicle for the present work, whereby the trainees' videos were assessed in terms of their competence using a recently revised rating scale (CTS-R). The inter-rater reliability of the supervisors' ratings was assessed both before and after the administration of a teaching package designed to improve the reliability in the use of the CTS-R. The results showed an improvement in the inter-rater agreement of five out of 12 items of the scale. The overall agreement also improved significantly following training.