Related Experiment Video
Updated: Jan 26, 2026
Establishing a Competing Risk Regression Nomogram Model for Survival Data
Published on: October 23, 2020
Judging clinical competence using structured observation tools: A cautionary tale
Anthony D Roth1, Pamela Myles-Hooton2, Amanda Branson2
1Research Department of Clinical, Educational and Health Psychology, University College London, London, UK.
Background:
One method for appraising the competence with which psychological therapy is delivered is to use a structured assessment tool that rates audio or video recordings of therapist performance against a standard set of criteria.
Aims:
The present study examines the inter-rater reliability of a well-established instrument (the Cognitive Therapy Scale - Revised) and a newly developed scale for assessing competence in CBT.
Method:
Six experienced raters working independently and blind to each other's ratings rated 25 video recordings of therapy being undertaken by CBT therapists in training.
Results:
Inter-rater reliability was found to be low on both instruments.
Conclusions:
It is argued that the results represent a realistic appraisal of the accuracy of rating scales, and that the figures often cited for inter-rater reliability are unlikely to be generalizable outside the specific context in which they were achieved. The findings raise concerns about the use of these scales for making summative judgements of clinical competence in both educational and research contexts.
Related Concept Videos
02:30Naturalistic Observations
01:23Actor-Observer Effect
01:12Observational Learning
01:11Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One...
01:08Data Collection by Observations
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
02:34Structural Isomerism
Isomers are different chemical species that have the same chemical formula. Structural isomerism of coordination compounds can be divided into two subcategories, the linkage isomers and coordination-sphere isomers.
Linkage isomers occur when the coordination compound contains a ligand that can bind to the transition metal center through two different atoms. For example, the CN− ligand can bind through the carbon atom or through the nitrogen atom. Similarly, SCN− can...