Multiplying Alpha: When Statistical Tests Compound in Sports Medicine Research

Area of Science:

• Sports Medicine
• Exercise Science
• Statistical Rigor

Background:

• Multiple statistical inferences increase the risk of Type I errors (false positives).
• Family-wise error rate (FWER) and experimental-wise error rate (EWER) corrections are inconsistently applied in sports medicine and exercise science.
• Uncorrected errors can lead to spurious findings and unreliable research conclusions.

Purpose of the Study:

• To highlight the critical issue of uncorrected FWER in sports medicine and exercise science research.
• To demonstrate the potential for spurious findings due to inadequate statistical error control.
• To emphasize the necessity of robust statistical methodologies for evidence-based practice.

Main Methods:

• Analysis of over 67 million regression models to illustrate the impact of FWER.
• Comparison of statistically significant findings (p<0.05) against expected false-positive rates.
• Examination of the consequences of failing to apply FWER corrections.

Main Results:

• Approximately 4.4% of models (3 million) showed statistical significance, consistent with the expected false-positive rate.
• This highlights the inflated risk of Type I errors when FWER is not addressed.
• The study underscores the prevalence of potential spurious findings in the absence of correction.

Conclusions:

• Rigorous statistical methods, including FWER control, are essential in sports medicine and exercise science.
• Solutions such as preregistration, false discovery rate control, and Bayesian approaches are recommended.
• Failure to correct for statistical errors can mislead clinical decision-making and potentially harm patients.