The Long Way From α-Error Control to Validity Proper: Problems With a Short-Sighted False-Positive Debate

Area of Science:

• Behavioral Science
• Empirical Science
• Psychology

Background:

• Growing awareness of inflated effect sizes and false-positive errors in behavioral science.
• Concerns regarding nonreplicable findings, data fabrication, and poor empirical practices.
• Critique of an isolated focus on false positives as potentially misleading.

Purpose of the Study:

• To argue that false negatives represent a more significant problem in current behavioral science.
• To highlight the limitations of statistical rigor when alternative hypotheses are not adequately tested.
• To advocate for a scientific culture that prioritizes strong inference over publication standards.

Main Methods:

• Analysis of inductive reasoning principles (Wason, 1960).
• Theoretical examination of the consequences of failing to test alternative hypotheses.
• Conceptual comparison of scientific cultures focused on strong inference versus publication standards.

Main Results:

• An overemphasis on false positives can obscure the more detrimental issue of false negatives.
• Failure to generate and test alternative hypotheses can lead to significant theoretical errors.
• Rigorous statistical testing of focal hypotheses cannot rectify errors stemming from neglected alternatives.

Conclusions:

• False negatives pose a more serious threat to scientific progress in behavioral science than false positives.
• A scientific culture that rewards strong inference and the testing of alternative hypotheses is essential for advancement.
• Shifting focus from merely tightening publication standards to fostering robust theoretical exploration is recommended.