Statistics in ophthalmology revisited: the (effect) size matters

Area of Science:

• Biomedical Sciences
• Statistics
• Medical Research

Background:

• The association between biomedical sciences and mathematics is fundamental.
• Null hypothesis significance testing (NHST) has long been central to statistical data analysis in medicine.
• Over-reliance on p-values has led to a dichotomous perception of reality and biased scientific reasoning.

Purpose of the Study:

• To critically evaluate the limitations of p-values in interpreting scientific experiments.
• To advocate for a more nuanced approach to statistical analysis in biomedical research.
• To emphasize the importance of effect sizes and reproducibility for enhancing scientific value.

Main Methods:

• Critical analysis of the role and interpretation of p-values in NHST.
• Discussion of alternative and complementary statistical approaches.
• Highlighting the significance of effect size and confidence intervals.
• Emphasis on reproducible research methodologies.

Main Results:

• P-values offer probability under a specific null hypothesis, not absolute certainty or population characteristics.
• Uncritical trust in p-values can lead to unjustified conclusions and flawed guidelines in scientific publications.
• Effect size, determined a priori, provides added value and aids in comparing research works.

Conclusions:

• Statistical significance (p-value) should not be equated with phenomenon nature or population characteristics.
• Investigators must interpret results critically within a broader context, using statistics as a tool for discussion, not conclusion.
• An ecumenical approach, incorporating effect sizes, confidence limits, and reproducible multi-center studies, is crucial for advancing scientific validity.