Understanding p-values and significance.

Area of Science:

• Statistics
• Biostatistics
• Scientific Methodology

Background:

• P-values and effect sizes are crucial for evaluating experimental outcomes.
• Interpretation of these metrics can be compromised by various forms of selection bias.
• Common sources of bias include multiple hypothesis testing and post-hoc data selection.

Purpose of the Study:

• To provide a clear introduction to the appropriate use of p-values.
• To guide non-specialists in understanding and applying p-values correctly.
• To identify and caution against problematic practices in statistical analysis.

Main Methods:

• Review of statistical principles for hypothesis testing.
• Explanation of selection bias and its impact on p-value interpretation.
• Guidance on best practices for data analysis and result selection.

Main Results:

• Identification of specific scenarios where p-value interpretation is vulnerable to bias.
• Demonstration of how multiple testing and informal data selection can invalidate results.
• Outline of principled approaches to using p-values effectively.

Conclusions:

• Adherence to principled statistical practices is essential for valid interpretation of experimental results.
• Awareness and avoidance of selection bias are critical for researchers.
• Proper use of p-values enhances the reliability and credibility of scientific findings.