Null hypothesis significance testing. On the survival of a flawed method

Area of Science:

• Psychology
• Scientific methodology
• Statistical inference

Background:

• Null hypothesis significance testing (NHST) is a foundational statistical method in scientific psychology.
• NHST has faced persistent criticism regarding its logical validity and inferential capabilities.
• Despite challenges, NHST remains a dominant tool for inductive reasoning in research.

Purpose of the Study:

• To review and analyze the criticisms leveled against NHST.
• To examine the defenses and pragmatic justifications for NHST's continued use.
• To explore the underlying Bayesian assumptions in both critiques and defenses of NHST.

Main Methods:

• Literature review of criticisms and defenses of NHST.
• Analysis of the logical and pragmatic arguments surrounding NHST.
• Examination of implicit Bayesian assumptions in the NHST debate.

Main Results:

• Criticisms of NHST focus on the logical validity of inferences.
• Defenses of NHST emphasize its practical utility in research.
• Both proponents and opponents of NHST often rely on implicit Bayesian principles.

Conclusions:

• Making Bayesian assumptions explicit is crucial for addressing NHST's limitations.
• The core challenge for NHST and all inductive systems is ensuring replicability.
• Understanding the role of Bayesian reasoning can lead to more robust scientific inference.