Properties of the Achievement of Therapeutic Objectives Scale (ATOS): a Generalizability Theory study

The psychometric properties of the process instrument Achievement of Therapeutic Objectives Scale (ATOS) were examined in this study. Generalizability Theory (GT) was used to evaluate variability attached to several possible sources of error. A random sample of 24 psychotherapy sessions was selected from a larger RCT trial including Cluster C patients and rated in different rating conditions (rating all scales versus rating a subgroup of the scales). Two G-study designs were used to compute variance components and generalizability coefficients. The results provided evidence that the ATOS is sensitive to differences among patients and to differences among subscales within patients (i.e., to different constructs within the ATOS). Rating condition contributed much to variability in scores when the ATOS scales were examined separately, but this variability was negligible when all scales were included in the same analysis. There was little variability due to raters, which indicates that adequately trained raters are able to apply the ATOS without contributing to measurement error.