Related Experiment Video
Updated: Jun 13, 2025

11:18
Using the Threat Probability Task to Assess Anxiety and Fear During Uncertain and Certain Threat
Published on: September 12, 2014
15.2K
Examining uncertainty in journal peer reviewers' recommendations: a cross-sectional study
Adrian Barnett1, Liz Allen2,3, Adrian Aldcroft4
1School of Public Health & Social Work, Queensland University of Technology, Brisbane, Australia.
Royal Society Open Science
|September 12, 2024
Summary
Scientific peer review is inconsistent. A study found 23% of reviewers experience uncertainty, impacting journal article recommendations. This variability is common in scientific publishing.
Area of Science:
- Scientific publishing
- Scholarly communication
Background:
- The peer review process is crucial for scientific publishing but faces criticism for inconsistency.
- Inconsistency in peer review stems from variations in reviewer expertise, training, and experience.
- Reviewer uncertainty, where a reviewer wavers between multiple recommendations, contributes to decision variability.
Purpose of the Study:
- To quantify the extent of within-reviewer uncertainty in the peer review process.
- To investigate factors associated with reviewer uncertainty, such as reviewer demographics and manuscript type.
Main Methods:
- Utilized post-review surveys at three journals to assess reviewer uncertainty.
- Asked reviewers to assign confidence percentages to all possible recommendations beyond their stated decision.
- Analyzed survey data to determine the prevalence of uncertainty and associated factors.
Main Results:
- Twenty-three percent of peer reviewers reported experiencing uncertainty in their recommendations (95% CI 19-27%).
- Reviewer uncertainty was associated with female reviewers at one journal.
- Protocol papers were associated with increased reviewer uncertainty at one journal.
Conclusions:
- Reviewer uncertainty is a common phenomenon in the peer review of journal articles.
- This within-reviewer variability is a significant component of the overall inconsistency observed in peer review recommendations.
- Understanding reviewer uncertainty can inform strategies to improve the reliability and consistency of scientific peer review.
Related Concept Videos
Propagation of Uncertainty from Random Error
656
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
656
Self-Discrepancy Theory
18.3K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.3K
Proofreading
53.9K
Overview
53.9K
Uncertainty: Overview
529
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
529
Cross-reactivity
31.0K
Overview
31.0K
The Scientific Method
225.2K
The scientific method is a detailed, empirical problem-solving process used by biologists and other scientists. This iterative approach involves formulating a question based on observation, developing a testable potential explanation for the observation (called a hypothesis), making and testing predictions based on the hypothesis, and using the findings to create new hypotheses and predictions.
Generally, predictions are tested using carefully-designed experiments. Based on the outcome of these...
Generally, predictions are tested using carefully-designed experiments. Based on the outcome of these...
225.2K

