Simple solution to a common statistical problem: interpreting multiple tests

The misinterpretation of the results of multiple statistical tests is an error commonly made in scientific literature. When testing several outcome variables simultaneously, many researchers declare a statistically significant result for each test having a P value of <0.05, for example. This approach ignores the fact that, based on a probability result called the Bonferroni inequality, the risk of incorrectly declaring as significant > or =1 test result increases with the number of tests conducted. The implication of this practice is that many scientific results are presented as statistically significant when the underlying data do not adequately support such a claim (sometimes referred to as false-positive results). Although the sequentially rejective Bonferroni test is well known among statisticians, it is not used routinely in scientific literature.

The intent of this article was to increase the awareness and understanding of the sequentially rejective Bonferroni test, thereby expanding its use.

This article describes the statistical problem and demonstrates how the use of the sequentially rejective Bonferroni test ensures that incorrect declarations of statistical significance for > or =1 test result are bounded by 0.05, for example.

The sequentially rejective Bonferroni test is an easily applied, versatile statistical tool that enables researchers to make simultaneous inferences from their data without risking an unacceptably high overall type I error rate.