Related Experiment Videos
Does mentoring new peer reviewers improve review quality? A randomized trial
Debra Houry1, Steven Green, Michael Callaham
1Department of Emergency Medicine, Emory University School of Medicine, Atlanta, GA, USA.
BMC Medical Education
|August 30, 2012
Summary
A structured mentoring program pairing new medical journal peer reviewers with senior mentors did not improve review quality. This study found no significant difference in subsequent review performance between mentored and unmentted reviewers.
Area of Science:
- Medical Education
- Scientific Publishing
- Peer Review
Background:
- Previous attempts to train medical journal peer reviewers yielded limited success.
- General and brief training interventions have not demonstrably improved review quality.
- A hypothesis proposed that manuscript-specific, extended mentoring could enhance reviewer performance.
Purpose of the Study:
- To evaluate the effectiveness of a structured mentoring program for new medical journal peer reviewers.
- To determine if pairing novice reviewers with senior mentors improves subsequent review quality.
- To assess the impact of a manuscript-specific, extended intervention on reviewer performance.
Main Methods:
- A four-year randomized controlled trial involving new peer reviewers for Annals of Emergency Medicine.
- Intervention group received standard materials plus mentoring; control group received only standard materials.
- Mentoring involved pairing new reviewers with senior reviewers for their first three manuscript reviews, with discussions via email or phone.
- Review quality was assessed using linear mixed effects models over time.
Main Results:
- The study included 490 manuscript reviews from 24 mentees and 22 control reviewers.
- Mean quality scores for the initial three reviews were comparable between groups (3.4 vs. 3.5).
- No significant differences were observed in the slopes of review score changes or other secondary performance measures.
Conclusions:
- A structured training intervention pairing new medical journal peer reviewers with senior mentors did not enhance the quality of their subsequent reviews.
- Extended, manuscript-specific mentoring did not lead to improved reviewer performance compared to standard training.
- The findings suggest that current structured mentoring approaches may not be sufficient to improve medical journal peer review quality.
Related Concept Videos
01:43Proofreading
Overview
00:46Feedback Inhibition
Biochemical reactions are occurring constantly in cells, converting starting substances to different products, usually with the help of enzymes that speed the reactions. Without enzymes, it would take far too long for most reactions to occur to be useful to the cell!
01:27Halo Effect
The halo effect is a cognitive bias in which an individual's overall impression influences judgments about their specific traits. This psychological phenomenon leads people to associate positive characteristics with those they perceive as generally good and negative characteristics with those they view as bad. This effect is particularly influential in social perception, professional evaluations, and decision-making processes.The Psychological Basis of the Halo EffectThe halo effect is rooted...
01:13Randomized Experiments
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
02:07Blind Procedures
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which child was...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...