A logical analysis of null hypothesis significance testing using popular terminology

Area of Science:

• Statistics
• Statistical Inference
• Scientific Methodology

Background:

• Null Hypothesis Significance Testing (NHST) is a widely used statistical method despite ongoing criticism.
• Traditional NHST explanations often equate the alternative hypothesis (H_A) with the research hypothesis (H_T).
• This study analyzes the logical structure of NHST using propositional calculus, focusing on common textbook presentations.

Purpose of the Study:

• To critically examine the internal logic of Null Hypothesis Significance Testing (NHST) as commonly presented to non-statisticians.
• To clarify the precise conclusions that can be drawn from a statistically significant result within the NHST framework.
• To highlight the need for greater transparency regarding the assumptions and premises used in NHST.

Main Methods:

• Employed propositional calculus to analyze the logical structure of NHST.
• Examined the definitions of the null hypothesis (H₀) and alternative hypothesis (H_A) in the context of comparing two sample group means.
• Analyzed the scope and limitations of P-values and test statistic probability distributions.

Main Results:

• The testable null hypothesis (H₀) in NHST is defined as the comparison of means being equal AND the observed difference being due to chance.
• The alternative hypothesis (H_A) is the logical negation of H₀, encompassing multiple possibilities beyond the research hypothesis (H_T).
• A statistically significant result (P-value < α) only supports the conclusion that findings are not solely due to chance.

Conclusions:

• The popular textbook definitions of H₀ and H_A in NHST differ from their logical implications.
• Achieving statistical significance does not automatically validate the research hypothesis (H_T); additional premises are required.
• The interpretation of Type I and Type II errors, and statistical power, needs careful consideration as they do not inherently refer to H_T.