Common measures and analytic techniques provide flawed assessments of pain: modeled data, and hip replacement study

To examine commonly used measures and analytic techniques of pain outcomes, using (1) a synthetic model, and (2) a cohort of patients who underwent total hip replacement.

(1) A synthetic data set was constructed with 110 visual analog scale (VAS) values, 10 for each integer from zero to 10. Random noise was added to simulate measurement variations. Drift through time and a therapeutic trial were simulated. (2) Eighty-six patients were studied before and a mean of 17 months after total hip replacement. Assessments included a VAS pain scale, the Western Ontario and McMaster Universities Osteoarthritis Index, Harris Hip Score, and SF-36 scores.

The clinical study mirrored the model. Correlation coefficients among treatment differences measured by the pain subscales of 4 instruments varied from 0.53 to 0.22. Floor effects obscured benefit. "Percentage improvement" created a directional bias, and had a hyperbolic distribution that invalidated means, variances, and related statistics. The best outcomes were undervalued when postoperative pain measures approached zero. Standardized means enabled pooling of data from the different instruments and facilitated measurement of the variations due to treatment, methods, and subjects, and other factors.

Outcome measures and analytic techniques are often flawed because of floor and ceiling effects, non-normal distributions, and other problems. Outcomes expressed as ""percentage improvement" are inappropriate; changes should be reported in the observed units. Revisions of standard outcome measures to relate pain with activity can better document outcomes, especially favorable results.