Why the resistance to statistical innovations? A comment on Sharpe (2013)

Area of Science:

• Psychology
• Quantitative Methods
• Statistical Analysis

Background:

• Substantive researchers show resistance to adopting newer statistical methods.
• Previous work focused on communication as a means to reduce this resistance.
• The impact of statistical innovations varies significantly across different subareas of psychology.

Purpose of the Study:

• To examine the resistance to replacing null hypothesis significance testing (NHST) with effect sizes and/or confidence intervals.
• To analyze the suitability of this statistical shift in research areas where only effect direction is meaningful.
• To explore how overgeneralization of criticisms and lack of tailored recommendations contribute to resistance.

Main Methods:

• Commentary on existing literature and statistical practices.
• Analysis of the implications of proposed statistical reforms for specific research types.
• Case study using research in social psychology, specifically priming effects and terror management theory.

Main Results:

• The movement to replace NHST may not be universally applicable or beneficial for all research areas.
• Research where only effect direction is meaningful, and specific effect sizes cannot be predicted, presents challenges for the proposed reforms.
• Overgeneralization of NHST criticisms and failure to tailor recommendations contribute to researcher resistance.

Conclusions:

• Statistical reformers need to consider the specific needs and methodologies of different psychological subareas.
• Tailored recommendations and improved communication are essential for the successful adoption of new statistical methods.
• Acknowledging the limitations of proposed reforms in certain research contexts can reduce resistance to change.