Suggested statistical standards for NTT manuscripts: notes from two of your reviewers

The purpose of a scientific paper in this journal is to persuade the reader of some important or potentially important facts. For a reader to be persuaded, first the manuscript reviewers must be persuaded, and if the manuscript involves statistical reasoning, at least one of those reviewers is likely to be a statistician. This invited article, by two long-time reviewers for Neurotoxicology and Teratology (NTT) who are also contributors of statistical papers, surveys some of the principles that render a manuscript more persuasive or less persuasive in our eyes. These principles are overwhelmingly not statistical but logical. For one typical NTT manuscript theme, the relation between some toxic exposure and one or more negative outcomes in humans, the aspects of manuscripts we scrutinize most closely include biological plausibility, dose-response relationships, breadth of evidence, adjustments for measurement bias, attention to assumptions and scatterplots in the search for confounds, and, in general, a sincere attempt to enunciate and then refute plausible hypotheses rival to the one the investigators prefer. The literature of excellent studies in other fields provides ample instances of good practice in these matters; we review it in those fields for applications in ours. Formal statistical significance testing plays almost no role in the most persuasive papers. In particular, findings that appear only after "adjustment for covariates" are never considered credible by these reviewers; we explain our reasons at length, and suggest alternatives.