Manuscript peer review lacks an empirically derived definition: a scoping review of 139 sources identifies 37...

To summarize empirical definitions and characteristics of manuscript peer review in the literature.

Scoping review conducted according to JBI methodology. We searched eight bibliographic databases and 76 grey-literature sources for English-language publications from 1986 onward addressing manuscript peer review in biomedical and health sciences, social sciences, economics, and healthcare-aligned industries. Title/abstract and full-text screening, empirical-source identification, and study-characteristic extraction were supported by an artificial intelligence-enabled workflow with human verification. Extracted empirically derived definitions and characteristics were coded, deduplicated, and thematically synthesized..

Of 4,181 eligible sources, 139 reported findings informed by empirical studies. All 139 sources described one or more characteristics of manuscript peer review, but only one provided an empirically derived definition. We identified 37 unique characteristics, organized into six themes: function, motivation, process, assessment, characteristics, and shortcomings. The findings show that manuscript peer review is commonly described in terms of evaluating research quality and reporting, supporting editorial decision-making and manuscript improvement, and involving expectations of constructive, timely, professional, critical, principled, and confidential review. Empirical sources also identified substantial limitations, including inconsistent standards, unregulated discretion, limited impact on reporting quality, delays, reviewer shortages and workload, weak incentives, power imbalances, ethical concerns, and bias.

This analysis identified a set of potential characteristics of manuscript peer review from empirical sources. These characteristics will be presented to Delphi study participants to support the development of a consensus definition of peer review. Achieving consensus in this area is expected to improve conceptual clarity, facilitate more consistent evaluation and comparison of peer review across fields, and inform future efforts to strengthen transparency and reform in scholarly publishing.

Peer review is the process in which people with relevant expertise assess a scholarly manuscript before, or sometimes after, it is published. It is intended to help editors decide whether a manuscript is suitable for publication and to help authors improve their work. Peer review is widely used across scholarly publishing, including research relevant to health care, but there is no single internationally accepted definition of what manuscript peer review is. We conducted a scoping review to identify how manuscript peer review has been defined and described in published research. A scoping review maps the available evidence on a topic rather than assessing whether one approach is better than another. We searched eight research databases and 76 grey-literature sources, such as websites and preprint platforms. We included English-language sources published from 1986 onwards that described manuscript peer review in biomedical and health sciences, social sciences, economics, and healthcare-related industries. We identified 4,181 sources that described peer review. Of these, 139 contained information based on empirical research, meaning that the information came from a study, review, or other formal investigation. Only one empirical study provided a formal definition of peer review. However, the studies identified 37 characteristics of the process. These were grouped into six areas: the purpose of peer review; reasons people participate; how the process is organized; what reviewers assess; features that characterize peer review; and problems or limitations of current peer-review practices. Our findings show that peer review is commonly described as a way to assess aspects of a manuscript such as its methods, reporting, relevance, and overall quality. At the same time, the literature describes important challenges, including inconsistent processes, potential bias, limited transparency, reviewer workload, and uncertainty about how well peer review achieves its intended goals. These findings will inform a future international consensus process involving people with different perspectives, including researchers, journal editors, statisticians, funders, policymakers, and patients and members of the public. The goal is to develop a clearer and more widely accepted definition of manuscript peer review. Such a definition could make peer review easier to study, compare, evaluate, and improve across scholarly publishing.