Toward evidence-based medical statistics. 1: The P value fallacy

Area of Science:

• Medical Research
• Statistical Inference
• Quantitative Data Analysis

Background:

• Modern medical research interpretation often lacks formal integration of biological understanding and prior research.
• This gap impacts the reliability of conclusions drawn from quantitative results.
• The standard statistical approach, using P values and hypothesis tests, creates this issue.

Purpose of the Study:

• To highlight the limitations of current statistical methods in medical research interpretation.
• To introduce the P value fallacy and its adverse effects on scientific inference.
• To propose the Bayes factor as a superior measure for evidence integration.

Main Methods:

• Review of the statistical debate surrounding P values and hypothesis testing.
• Analysis of the P value fallacy and its implications.
• Introduction to the principles and application of Bayes factors.

Main Results:

• The standard statistical approach creates an illusion of error rates without external context.
• P values incorrectly conflate long-run experimental outcomes with single-result evidence.
• Bayes factors offer a method to separate long-run behavior from evidential strength.

Conclusions:

• Relying solely on P values hinders the integration of crucial background knowledge in medical research.
• The P value fallacy leads to potentially unreliable scientific conclusions.
• Bayes factors provide a more robust framework for statistical inference by incorporating prior knowledge.