Psychological treatment outcomes in routine NHS services: what do we mean by treatment effectiveness?

The question of how effective therapies are in routine practice is crucial. The answer depends on how we define effectiveness. Both the definition of who was treated and the index chosen to represent outcome can affect estimates dramatically.

We used data from the Clinical Outcomes in Routine Evaluation (CORE) Primary Care National Practice-Based Evidence database-2005 on the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) from 33,587 patients and examined rates of improvement in psychological therapies in UK National Health Service (NHS) primary care services using various definitions of effectiveness.

We constructed successively more restrictive sub-samples of patients, including practice-based analogues of several types of intent-to-treat (ITT) groups and completer groups. We focussed on patients scoring above clinical cut-off at intake, but we also considered samples that included sub-clinical patients. We assessed two types of recovery rates, improvement rates, mean pre-post change, and pre-post effect sizes for each sub-sample.

There was wide variation in the overall effectiveness of treatments as a function of which subset of data was considered and which specific criterion of recovery rate was adopted. Recovery rates and pre-post effect sizes ranged from 19% to 65% and 0.60 to 1.95, respectively.

Because estimates of effectiveness could have significant policy implications, clarity on the meanings of the differing constructions is essential.