Related Experiment Video
Updated: Jun 8, 2026
10:39
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Implementation of a journal peer reviewer stratification system based on quality and reliability
Steven M Green1, Michael L Callaham
1Department of Emergency Medicine, Loma Linda University, CA 92354, USA. steve@viridissimo.com
Annals of Emergency Medicine
|October 16, 2010
Summary
Implementing a tiered reviewer system improved journal peer review efficiency by stratifying reviewers based on quality and reliability, leading to fewer late reviews and better response rates.
Area of Science:
- Medical Journal Publishing
- Scientific Peer Review Process
Background:
- Annals of Emergency Medicine faced challenges with a large, variable reviewer pool.
- Inconsistent reviewer quality and reliability impacted journal efficiency.
Purpose of the Study:
- To test the hypothesis that a tiered reviewer stratification system could enhance journal efficiency.
- To enable editors to target high-quality reviewers more effectively.
Main Methods:
- A 3-tiered hierarchic classification system for reviewers was implemented in 2003.
- Reviewers were stratified based on predefined quality and reliability criteria.
- Editors focused review invitations on the top performance tier.
Main Results:
- Late reviews decreased from 32% (2002) to 13% (2009).
- Unused reviewers decreased from 59% (2002) to 28% (2009).
- On-time reviews from top-tier invitations increased from 37% (2002) to 48% (2009).
Conclusions:
- The tiered reviewer stratification system was successfully implemented.
- The system demonstrably improved the efficiency of the journal's peer review process.
Related Concept Videos
01:19Quality Assurance
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
01:46Types of Selection
Natural selection influences the frequencies of particular alleles and phenotypes within populations in several different ways. Primarily, natural selection can be directional, stabilizing, or disruptive. Directional selection favors one extreme trait and shifts the population towards that phenotype while selecting against individuals displaying alternate traits. Stabilizing selection favors an intermediate trait with a narrow range of variation. Deviation from the optimal phenotype towards an...
01:05Quality Control
Quality control is one of the three cyclical quality assurance activities that help keep a system under statistical control. Typical quality control activities include creating quality control charts, conducting proficiency testing, and documenting and archiving results.
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...
01:16Stratified Sampling Method
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:02Quantifying and Rejecting Outliers: The Grubbs Test
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This number is...