Related Experiment Video
Updated: Aug 6, 2026
Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Simulated changes in systematic review screening sensitivity and specificity associated with varying co-screening
Shaun Prentice1,2, Ben Mundy2, Caitlin Prentice3
1School of Psychology, College of Education, Behavioural and Social Sciences, Adelaide University, Adelaide, SA, Australia.
Objective:
The objective of this study was to examine how sensitivity and specificity of title/abstract screening decisions change as the proportion of records that are co-screened is varied.
Introduction:
With an accelerating volume of literature being published, the demand for systematic reviews is increasing. To maintain the feasibility of conducting reviews in a timely fashion, efficiencies must be found in the review process that do not compromise their rigor. Current guidelines recommend that all records be dual-screened. Less is known about the impact of co-screening a subset of records on screening accuracy.
Methods:
Eight reviewers independently screened the titles and abstracts from a published systematic review and meta-analysis. Using their decisions, simulations were run to generate permutations of reviewer combinations in 2 formats: dual screening (ie, primary reviewer screened all records, secondary reviewer screened a portion, third reviewer mediated disagreements) and 3-way screening (ie, primary reviewer, secondary reviewer, and mediator roles were equally divided between 3 individuals). Within each permutation, further simulations were created so that the secondary reviewer screened increasing proportions of records in 5% increments. Decisions were compared against the final full-text decisions used in the published review to determine sensitivity and specificity of decisions for each permutation, at each increment. The relationship between sensitivity/specificity and percentage of co-screening was assessed using linear mixed-effects models.
Results:
All reviewers screened 3891 potentially eligible records, of which 80 were included in the final review. Sensitivity (median range: 93.3%-96.3%) and specificity (median range: 87.7%-89.6%) were stable across different proportions of co-screening, for both dual- and 3-way screening strategies. Sensitivity was higher in reviewer groupings where at least one reviewer had subject matter experience. Specificity tended to be higher when the primary reviewer had experience in conducting a systematic review.
Discussion:
Our results suggest that complete co-screening of title/abstract records does not appreciably improve sensitivity or specificity. Rather, even low levels of co-screening across dual- and 3-way screening can produce high accuracy. Reviewer experience in subject matter and in conducting a systematic review, however, appear to be influential. These results are based on simulations and permutations from a single review so should be used as a guide only.
Conclusion:
Co-screening a proportion of title/abstracts in a systematic review appears to produce comparable sensitivity and specificity to co-screening all records. Proportional co-screening may therefore represent an opportunity for enhancing efficiency in the systematic review process.
Review Registration:
https://osf.io/9feuh.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:24Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...