A comparison of reports from referees chosen by authors or journal editors in the peer review process

The aim was to analyse the peer review process by comparing reports produced by referees selected by journal editors, with those of referees selected by the authors of a scientific manuscript. Some 104 consecutive papers from the UK submitted to the British Journal of Surgery (BJS) were included. Of these, 102 were reviewed blind both by referees chosen by the journal editors, and referees chosen by the paper's principal author. Manuscripts were marked using a standard sheet for four basic aspects: originality, clinical/scientific importance, clarity and analysis; a final overall recommendation about possible publication was given. The time taken and the number of completed referee reports were similar in each group. Referees chosen by the BJS editors were more critical (scored higher) of the submitted articles. Mean scores for all domains were higher than for authors' referees, significantly for scientific importance (p = 0.009) and decision to publish (p = 0.029). In conclusion, reports produced by referees selected by BJS editors were more critical than those chosen by authors of the papers. Authors might argue that this reduced their chance of publication but constructive criticism might improve the final article and assist editors to make decisions about acceptance or rejection.