Does tuition for journal referees work? A quantitative evaluation of a half-day tuition course

Most consultants participating as referees in the peer review process of papers submitted to scholarly journals have had no training or tuition. This study attempted to evaluate the effect on reviewing of a half-day course held at the Royal Society of Medicine.

Registered consultant delegates were sent two 'doctored' papers, a case report and an original paper, well before the meeting to review at home using the standard computerized score sheet issued with referee requests by Colorectal Disease. At the start of the meeting the scores were entered into a computer as 'Before'. After each paper had been presented and then discussed, it was re-marked to give the 'After' score. The Before and After scores were compared with the post-meeting feedback forms.

The Before and After scores provided by the participants for the two papers each reviewed were not significantly different for the questions relating to the publication/rejection decision. The Before score was higher than the After score for questions relating to the Abstract, Introduction and Method sections. Feedback forms regarding the tuition were universally positive and appreciative.

Consultants already have the expertise to decide whether a paper should be rejected. The study day appears to give an additional insight that may change an initial opinion. In general a paper scored before the meeting was scored lower after it was presented and discussed at the meeting (the tuition).