Can Sex and Seniority Predict the Quality of a Journal Reviewer's Manuscript Critique?

To evaluate reviewers' timeliness and review quality for the International Journal of Radiation Oncology, Biology, Physics (IJROBP) by sex and seniority.

The IJROBP editorial office provided data on 3962 individuals invited to review manuscripts from 2011 through 2014. We identified 1657 reviewers who had been invited to provide a review on at least 3 occasions during the study period and compared review timeliness and scoring between male and female reviewers. We confirmed the reviewers' sex after having unblinded their names based on our personal acquaintance with them and via an Internet search on their department websites. We then did a subset analysis of 124 US-based reviewers who had returned a "major revision" decision. We used the Review Quality Instrument (RQI) to rate their reviews. We used odds ratios and t tests to look for differences in mean RQI scores and factors that might be associated with quality-in particular, Hirsch indices (h indices) and year of first certification.

Of the 1657 reviewers of interest, 1245 (75.1%) were men and 412 (24.9%) were women. We found no statistically significant differences between men and women in the time to respond to invitations. There were no statistically significant differences in timeliness or review reminders based on sex. Our subset analysis showed no difference in quality (RQI scores) based on the reviewers' sex, h index, or year of first certification.

Women and men render reviews of equal quality regardless of seniority and h index, yet women have been invited less frequently to review. This is likely because of the underrepresentation of women in radiation oncology. A more balanced academic population is needed to address this continuing disparity of women's representation in academic publishing.