Reviewer training for improving grant and journal peer review

Area of Science:

• Scientific publishing
• Research integrity

Background:

• Peer review is crucial for funding and publication decisions.
• Reviewer training aims to enhance peer review quality, but existing studies show inconsistent results.
• There is a lack of up-to-date systematic reviews on the effectiveness of reviewer training.

Purpose of the Study:

• To systematically evaluate the impact of peer reviewer training on the quality of both grant and journal peer review.

Main Methods:

• Conducted an extensive Cochrane search up to April 2022.
• Included 10 randomized controlled trials (RCTs) comparing training interventions with standard practices.
• Assessed primary outcomes of reporting completeness and error detection, with secondary outcomes including bibliometric scores and stakeholder assessments.

Main Results:

• Moderate-certainty evidence indicates reminder emails have little to no effect on reporting completeness.
• Low-certainty evidence suggests reviewer training slightly improves error detection but has little to no effect on stakeholder assessment of review quality.
• Very low to low-certainty evidence shows minimal impact on inter-reviewer agreement in grant peer review.

Conclusions:

• Current evidence from 10 RCTs suggests peer reviewer training yields little to no improvement in peer review quality.
• Future studies require larger participant numbers and a wider range of validated outcome measures.
• Ensuring validity and reliability of stakeholder assessment instruments is crucial for future research.