Related Experiment Video
Updated: Feb 9, 2026

In Vitro Rearing of Solitary Bees: A Tool for Assessing Larval Risk Factors
Published on: July 16, 2018
Three risk of bias tools lead to opposite conclusions in observational research synthesis
Josep-Maria Losilla1, Isabel Oliveras1, Juan A Marin-Garcia2
1Department of Psychobiology and Methodology of Health Sciences, Psychology Faculty, Universitat Autònoma de Barcelona, Carrer de la Fortuna, Edifici B, Bellaterra, Barcelona 08193, Spain.
Objectives:
The aim of this study was to assess the agreement and compare the performance of three different instruments in assessing risk of bias (RoB) of comparative cohort studies included in a health psychology meta-analysis.
Study Design And Setting:
Three tools were applied to 28 primary studies included in the selected meta-analysis: the Newcastle-Ottawa Scale, quality of cohort studies (Q-Coh), and risk of bias in nonrandomized studies of interventions (ROBINS-I).
Results:
Interrater agreement varied greatly from tool to tool. For overall RoB, 75% of the studies were rated as low RoB with the Newcastle-Ottawa Scale, 11% of the studies with Q-Coh, and no study was found to be at low RoB using ROBINS-I. No influence of quality ratings on the meta-analysis results was found for any of the tools.
Conclusion:
Assessing RoB using the three tools may lead to opposite conclusions, especially at low and high levels of RoB. Domain-based tools (Q-Coh and ROBINS-I) provide a more comprehensive framework for identifying potential sources of bias, which is essential to improving the quality of future research. Both further guidance on the application of RoB tools and improvements in the reporting of primary studies are necessary.
More Related Videos
04:14Facile Synthesis of Colloidal Lead Halide Perovskite Nanoplatelets via Ligand-Assisted Reprecipitation
Published on: October 1, 2019
10:11Fundus Photography as a Convenient Tool to Study Microvascular Responses to Cardiovascular Disease Risk Factors in Epidemiological Studies
Published on: October 22, 2014
Related Concept Videos
Confirmation Biases
Naturalistic Observations
Hindsight Biases
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Correspondence Bias
Self-Serving Bias