Avoiding pitfalls of correlation coefficients in the assessment of measurement instruments in rehabilitation research

To provide a practical guide on how to avoid the pitfalls of correlated correlation coefficients when comparing multiple instruments in rehabilitation research.

An observational study comparing a number of instruments measuring quality of life (QoL) compared with an external criterion.

Sixty-eight patients admitted to a rheumatology ward for intensive treatment of rheumatoid arthritis.

Patients completed three new (QoL) instruments and an established instrument before and after intensive treatment for rheumatoid arthritis.

Correlation coefficients together with their confidence intervals and a test for the difference between a set of correlated correlation coefficients for the change in the EuroQoL Quality of Life scale (EuroQoL), the World Health Organization Quality Of Life-Abbreviated version (WHOQoL-BREF) and the Quality of Life Profile (QLP) against the Stanford Health Assessment Questionnaire (HAQ).

Although the range of correlation between the new instruments and the external criterion was between -0.37 and -0.59 and suggested that one new instrument was far more responsive than the others,; an omnibus test for an overall difference could find no difference in responsiveness.

It is conceptually simple to use correlation coefficients to assess the properties of multiple instruments measured on the same subjects to find a 'best' instrument. However, proper interpretation of results when correlated correlation coefficients are calculated is complex. We recommend analysis includes: (a) that simple plots of the pairs of analysed variables are shown, (b) that simple linear model-fitting statistics, e.g., the R-squared statistic, accompany the plots, (c) that confidence intervals are presented for correlation coefficients, (d) that an omnibus statistical test for the difference between correlated correlation coefficients is presented, and (e) that normal model assumptions are tested.