Why Hypothesis Testers Should Spend Less Time Testing Hypotheses

Area of Science:

• Psychology
• Social Sciences
• Scientific Methodology

Background:

• The mindless use of null-hypothesis significance tests has rendered social science research on theories largely uninterpretable.
• Recent reforms in psychology have emphasized formalizing hypothesis testing procedures in response to the replication crisis.

Purpose of the Study:

• To argue that premature hypothesis testing hinders scientific progress.
• To advocate for a shift towards nonconfirmatory research activities to strengthen psychological theories.

Main Methods:

• Conceptual analysis of current psychological research practices.
• Discussion of Paul Meehl's critiques of null-hypothesis significance testing.
• Proposal for prioritizing theory-building activities before hypothesis testing.

Main Results:

• Current reforms risk forcing premature hypothesis testing, which is counterproductive.
• Nonconfirmatory research activities are crucial for developing informative hypothesis tests.
• Establishing a sound "derivation chain" between theory and test requires foundational work.

Conclusions:

• Psychological scientists need more time for concept formation, measure development, and establishing relationships between concepts.
• Incentivizing foundational research inputs is essential for developing strong, testable theories.
• Shifting focus to nonconfirmatory research can resolve issues in psychology's reform movement and fulfill Meehl's vision.