Reviewing scientific manuscripts: how much statistical knowledge should a reviewer really know?

In the sequel to their guidelines for reporting statistics in American Physiological Society journals, Curran-Everett and Benos highlighted that the initial guidelines of 2004 have had little effect on the statistical reporting practices of authors. In the present article, I suggest that the guidelines have also had little impact on both journal reviewers and editors. I present three cases of statistical reporting practices in which there appears to be considerable discrepancies between the author and reviewer and, moreover, inconsistencies between reviewers. I argue that for authors to comply with these guidelines, the initial challenge is to have a team of reviewers who are also willing to accept the unfamiliar. Indeed, the opinions of reviewers who are ill informed about relatively novel statistical methods and recommended reporting practices may have implications for the final editorial decision on the suitability of submitted manuscripts for publication.