Community Members as Reviewers of Medical Journal Manuscripts: a Randomized Controlled Trial

Community members may provide useful perspectives on manuscripts submitted to medical journals.

To determine the impact of community members reviewing medical journal manuscripts.

Randomized controlled trial involving 578 original research manuscripts submitted to two medical journals from June 2018 to November 2021.

Twenty-eight community members who were trained, supervised, and compensated.

A total of 289 randomly selected control manuscripts were reviewed by scientific reviewers only. And 289 randomly selected intervention manuscripts were reviewed by scientific reviewers and one community member. Journal editorial teams used all reviews to make decisions about acceptance, revision, or rejection of manuscripts.

Usefulness of reviews to editors, content of community reviews, and changes made to published articles in response to community reviewer comments.

Editor ratings of community and scientific reviews averaged 3.1 and 3.3, respectively (difference 0.2, 95% confidence interval [CI] 0.1 to 0.3), on a 5-point scale where a higher score indicates a more useful review. Qualitative analysis of the content of community reviews identified two taxonomies of themes: study attributes and viewpoints. Study attributes are the sections, topics, and components of manuscripts commented on by reviewers. Viewpoints are reviewer perceptions and perspectives on the research described in manuscripts and consisted of four major themes: (1) diversity of study participants, (2) relevance to patients and communities, (3) cultural considerations and social context, and (4) implementation of research by patients and communities. A total of 186 community reviewer comments were integrated into 64 published intervention group articles. Viewpoint themes were present more often in 66 published intervention articles compared to 54 published control articles (2.8 vs. 1.7 themes/article, difference 1.1, 95% CI 0.4 to 1.8).

With training, supervision, and compensation, community members are able to review manuscripts submitted to medical journals. Their comments are useful to editors, address topics relevant to patients and communities, and are reflected in published articles.

ClinicalTrials.gov NCT03432143.