Fisher, Neyman-Pearson or NHST? A tutorial for teaching data testing

Area of Science:

• Statistics
• Research Methodology
• Scientific Teaching

Background:

• Null Hypothesis Significance Testing (NHST) is widely used in behavioral, social, and biomedical research.
• Despite calls for reform, NHST remains entrenched in current research practices and education.
• The optimal time to influence statistical practices is during the teaching of hypothesis testing procedures.

Purpose of the Study:

• To provide a tutorial for teaching data testing procedures, including Fisher's tests of significance and Neyman-Pearson's tests of acceptance.
• To explain the combination of these approaches into the current NHST framework.
• To offer practical solutions for improving NHST for researchers who continue to use it.

Main Methods:

• Introduction to Fisher's approach to significance testing.
• Explanation of Neyman-Pearson's approach to acceptance testing.
• Analysis of the integration of these methods into NHST and proposed improvements.

Main Results:

• The tutorial clarifies the distinct origins and components of significance testing and acceptance testing.
• It highlights the incongruent combination leading to current NHST practices.
• Two compromise solutions for enhancing NHST are presented.

Conclusions:

• Understanding the historical development of hypothesis testing is crucial for effective statistical practice.
• The current NHST framework can be understood and potentially improved by examining its constituent parts.
• Educating researchers on different hypothesis testing theories can lead to more informed and robust data analysis.