Agreement between pre-post measures of change and transition ratings as well as then-tests

Different approaches have been developed for measuring change. Direct measurement of change (transition ratings) requires asking a patient directly about his judgment about the change he has experienced (reported change). With indirect measures of change, the patients' status is assessed at different time points and differences between them are calculated (measured change). When using the quasi-indirect approach ('then-test'), patients are asked after an intervention to rate their statuses both before the intervention as well as at the time of the enquiry. Associations previous studies have found between the different approaches might be biased because transition ratings are generally assessed using a single, general item, while indirect measures of change are generally based on multi-item scales. We aimed to quantify the agreement between indirect and direct as well as indirect and quasi-indirect measures of change while using multi-item scales exclusively. We explored possible reasons for non-agreement (present-state bias, recall bias).

We re-analysed a data set originally collected to investigate the prognostic validity of different approaches of change measurements. Patients from a 3-week inpatient rehabilitation programme for either cardiac or musculoskeletal disorders filled in health-status questionnaires (which included scales for sleep function, physical function, and somatisation) both at admission and at discharge. The patients were then randomised to receive either an additional transition-rating or then-test questionnaire at discharge.

Out of 426 patients, 395 (92.7%) completed all questionnaires. Correlation coefficients between indirect and quasi-indirect measures of change ranged from r = .60 to r = .71, compared to r = .37 to r = .48 between indirect and direct measures of change. Correlation coefficients between pre-test and retrospective pre-test (then-test) results ranged from r = .69 to r = .82, indicating a low level of recall bias. Pre-test variation accounted for a substantial amount of variance in transition ratings in addition to the post-test scores, indicating a low level of present-state bias.

Indirect and quasi-indirect measurements of change yielded comparable results indicating that recall bias does not necessarily affect quasi-indirect measurement of change. Quasi-indirect measurement might serve as a substitute for pre-post measurement under conditions still to be specified. Transition ratings reflect different aspects of change than indirect and quasi-indirect methods do, but are not necessarily biased by patients' present states.