Misuse of null hypothesis significance testing: would estimation of positive and negative predictive values improve...

Although generally misunderstood, the p value is the probability of the test results or more extreme results given H0 is true: it is not the probability of H0 being true given the results. To obtain directly useful insight about H0, the positive predictive value (PPV) and the negative predictive value (NPV) may be useful extensions of null hypothesis significance testing (NHST). They provide information about the probability of statistically significant and non-significant test outcomes being true based on an a priori defined biologically meaningful effect size. The present study explores the utility of PPV and NPV in an ecotoxicological context by using the frequently applied Daphnia magna reproduction test (OECD guideline 211) and the chemical stressor lindane as a model system. The results indicate that especially the NPV deviates meaningfully between a test design strictly following the guideline and an experimental procedure controlling for α and β at the level of 0.05. Consequently, PPV and NPV may be useful supplements to NHST that inform the researcher about the level of confidence warranted by both statistically significant and non-significant test results. This approach also reinforces the value of considering α, β, and a biologically meaningful effect size a priori.