Why the resistance to statistical innovations? A comment on Sharpe (2013)

Sharpe's (2013) article considered reasons for the apparent resistance of substantive researchers to the adoption of newer statistical methods recommended by quantitative methodologists, and possible ways to reduce that resistance, focusing on improved communication. The important point that Sharpe missed, however, is that because research methods vary radically from one subarea of psychology to another, a particular statistical innovation may be much better suited to some subareas than others. Although there may be some psychological or logistical explanations that account for resistance to innovation in general, to fully understand the resistance to any particular innovation, it is necessary to consider how that innovation impacts specific subareas of psychology. In this comment, I focus on the movement to replace null hypothesis significance testing (NHST) with reports of effect sizes and/or confidence intervals, and consider its possible impact on research in which only the direction of the effect is meaningful, and there is no basis for predicting specific effect sizes (and very large samples are rarely used). There are numerous examples of these studies in social psychology, for instance, such as those that deal with priming effects. I use a study in support of terror management theory as my main example. I conclude that the degree to which statistical reformers have overgeneralized their criticisms of NHST, and have failed to tailor their recommendations to different types of research, may explain some of the resistance to abandoning NHST. Finally, I offer suggestions for improved communication to supplement those presented by Sharpe. (PsycINFO Database Record