Experimental comparison of parametric versus nonparametric analyses of data from the cold pressor test

Parametric statistical methods are common in human pain research. They require normally distributed data, but this assumption is rarely tested. The current study analyzes the appropriateness of parametric testing for outcomes from the cold pressor test (CPT), a common human experimental pain test. We systematically reviewed published CPT studies to quantify how often researchers test for normality and how often they use parametric versus nonparametric tests. We then measured the normality of CPT data from 7 independent small to medium cohorts and 1 study of >10,000 subjects. We then examined the ability of 2 common mathematical transformations to normalize our skewed data sets. Lastly, we performed Monte Carlo simulations on a representative data set to compare the statistical power of the parametric t-test versus the nonparametric Wilcoxon Mann-Whitney test. We found that only 39% of published CPT studies (47/122) mentioned checking data distribution, yet 72% (88/122) used parametric statistics. Furthermore, among our 8 data sets, CPT outcomes were virtually always nonnormally distributed, and mathematical transformations were largely ineffective in normalizing them. The simulations demonstrated that the nonparametric Wilcoxon Mann-Whitney test had greater statistical power than the parametric t-test for all scenarios tested: For small effect sizes, the Wilcoxon Mann-Whitney test had up to 300% more power.

These results demonstrate that parametric analyses of CPT data are routine but incorrect and that they likely increase the chances of failing to detect significant between-group differences. They suggest that nonparametric analyses become standard for CPT studies and that assumptions of normality be routinely tested for other types of pain outcomes as well.