Related Experiment Video
Updated: May 15, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Testing a tool for assessing the risk of bias for nonrandomized studies showed moderate reliability and promising
Soo Young Kim1, Ji Eun Park, Yoon Jae Lee
1National Evidence-Based Healthcare Collaborating Agency, Seoul, Republic of Korea. hallymfm@gmail.com
Objectives:
To develop and validate a new risk-of-bias tool for nonrandomized studies (NRSs).
Study Design And Setting:
We developed the Risk of Bias Assessment Tool for Nonrandomized Studies (RoBANS). A validation process with 39 NRSs examined the reliability (interrater agreement), validity (the degree of correlation between the overall assessments of RoBANS and Methodological Index for Nonrandomized Studies [MINORS], obtained by plotting the overall risk of bias relative to effect size and funding source), face validity with eight experts, and completion time for the RoBANS approach.
Results:
RoBANS contains six domains: the selection of participants, confounding variables, the measurement of exposure, the blinding of the outcome assessments, incomplete outcome data, and selective outcome reporting. The interrater agreement of the RoBANS tool except the measurement of exposure and selective outcome reporting domains ranged from fair to substantial. There was a moderate correlation between the overall risks of bias determined using RoBANS and MINORS. The observed differences in effect sizes and funding sources among the assessed studies were not correlated with the overall risk of bias in these studies. The mean time required to complete RoBANS was approximately 10 min. The external experts who were interviewed evaluated RoBANS as a "fair" assessment tool.
Conclusions:
RoBANS shows moderate reliability, promising feasibility, and validity. The further refinement of this tool and larger validation studies are required.
Related Concept Videos
Reliability and Validity
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Bias in Epidemiological Studies
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Cochran's Q Test
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with data...
