An exploratory study of statistical assessment of papers published in the British Medical Journal

Statistical assessment of papers submitted to the British Medical Journal has increased to some 300 papers annually. The assessment produces a recommendation to the editor on each paper from a statistical viewpoint together with a completed checklist that indicates the quality of certain important features. This exploratory study was aimed at monitoring the process. It reports a comparison of checklist answers on 45 papers as originally submitted with those on the papers as subsequently published. Of the 45 papers, only 5 (11%) were considered statistically acceptable at submission, but this increased to 38 (84%) after publication. Revisions had not been made adequately in 4 of the 7 unsatisfactory published papers, and the 3 others were thought to be of dubious validity. A major omission from at least 28 papers was information on sample size calculations. It is concluded that statistical assessment is beneficial but that further efforts by authors and assessors could make it even more effective.