The P Value: What It Is and What It Is Not

Area of Science:

• Biostatistics
• Medical Research Methodology

Background:

• The P value is a common but frequently misunderstood statistical measure in biomedical literature.
• Its interpretation has evolved from Fisher's evidential framework to Neyman-Pearson's decision framework, leading to misconceptions.
• Over-reliance on the P = 0.05 threshold has resulted in misinterpretations, such as equating statistical significance with clinical importance.

Purpose of the Study:

• To review the historical development and conceptual underpinnings of the P value.
• To clarify the distinctions between evidential and decision-theoretic perspectives on P values.
• To discuss common misinterpretations and limitations of P value-based inference.

Main Methods:

• Historical review of the P value's evolution.
• Conceptual analysis of statistical inference frameworks.
• Case study illustration of P value implications.
• Discussion of consequences for reproducibility and statistical power.

Main Results:

• The P value is often misinterpreted as the probability of the null hypothesis being true.
• Threshold-based inference (P = 0.05) has limitations impacting reproducibility and interpretation.
• Statistical significance does not inherently imply clinical importance.

Conclusions:

• The P value can offer insights but should not be the sole basis for scientific inference.
• Complementary methods like effect size estimation with confidence intervals (CIs) are recommended.
• Transparent reporting of effect sizes, CIs, and contextual data enhances scientific interpretation and decision-making.