Invited Commentary: Can Issues With Reproducibility in Science Be Blamed on Hypothesis Testing?

Area of Science:

• Epidemiology
• Genetics
• Scientific Methodology

Background:

• Reproducibility is a significant challenge in scientific research.
• The culture of null hypothesis significance testing (NHST) is implicated in reproducibility issues.
• Selective attention to statistically significant findings can lead to biased effect estimates.

Purpose of the Study:

• To argue for the continued importance of statistical testing in scientific research.
• To address concerns about the "culture" of NHST and its impact on scientific reproducibility.
• To propose statistical testing as a necessary selection strategy in "innovative" research.

Main Methods:

• Revisiting the recent history of genetic epidemiology.
• Analyzing the role of null hypothesis significance testing in scientific bias.
• Evaluating the utility of statistical testing as a selection strategy.

Main Results:

• NHST, when selectively applied, can systematically bias effect estimates away from the null.
• In "innovative" research involving agnostic consideration of many factors, a selection strategy is needed.
• Statistical testing provides a vital tool for identifying potentially genuine findings.

Conclusions:

• Despite criticisms, statistical testing remains an essential component of the scientific toolkit.
• A selection strategy is necessary to identify promising findings in exploratory research.
• Retaining statistical testing aids in navigating the complexities of modern genetic epidemiology.