The use of confidence intervals in reporting orthopaedic research findings

Conflict between clinical importance and statistical significance is an important problem in medical research. Although clinical importance is best described by asking for the effect size or how much, statistical significance can only suggest whether there is any difference. One way to combine statistical significance and effect sizes is to report confidence intervals. We therefore assessed the reporting of confidence intervals in the orthopaedic literature and factors influencing this frequency. In parallel, we tested the predictive value of statistical significance for effect size. In a random sample of predetermined size, we found one in five orthopaedic articles reported confidence intervals. Participation of an individual trained in research methods increased the odds of doing so fivefold. The use of confidence intervals was independent of impact factor, year of publication, and significance of outcomes. The probability of statistically significant results to predict at least a 10% between-group difference was only 69% (95% confidence interval, 55%-83%), suggesting that a high proportion of statistically significant results do not reflect large treatment effects. Confidence intervals could help avoid such erroneous interpretation by showing the effect size explicitly.