[Effects on scale and confidence intervals as alternatives to p < 0.05]

Researchers and reviewers often use the conventional p < 0.05 as threshold in statistical tests. In many cases, however, the interpretation of p-values is incorrect.

To explain where the 5% norm originates, identify the interpretation problems that often arise and suggest some alternatives.

On the basis of recent literature we examine the meaning and origin of the p < 0.05 norm. We looked closely at entire articles and short reports in the Tijdschrift voor Psychiatrie, starting with the Jubilee issue of 2008, in order to find examples of methodological problems relating to the routine use of p-values.

We found several examples of the problematic use of p-values; these included the testing of a priori unlikely, or even impossible null hypotheses, the reporting of small effects calculations based on erroneous assumptions, and incorrect interpretations of statistical parameters and p-values.

Research in psychiatry, like research in other disciplines, attaches too much weight to p-values. Guidelines for authors should advise authors to focus explicitly on effect sizes, confidence intervals and the scale on which the results are presented.