Related Experiment Video
Updated: Feb 1, 2026

Systematic Assessment of Well-Being in Mice for Procedures Using General Anesthesia
Published on: March 20, 2018
Minor differences were found between AMSTAR 2 and ROBIS in the assessment of systematic reviews including both
Dawid Pieper1, Livia Puljak2, Marien González-Lorenzo3
1Institute for Research in Operative Medicine, Evidence-based Health Services Research, Faculty of Health, School of Medicine, Witten/Herdecke University, Ostmerheimer Str, Cologne 200 51109, Germany.
Objective:
To compare A Measurement Tool to Assess Systematic Reviews (AMSTAR 2) with a tool to assess risk of bias in systematic reviews (ROBIS) in terms of validity, reliability, and applicability.
Study Design And Setting:
We analyzed 30 systematic reviews (SRs) that included randomized and nonrandomized studies, with Cochrane and non-Cochrane SRs sampled in 1:1 ratio. Four reviewers assessed independently all 30 SRs with AMSTAR 2, followed by ROBIS. We calculated Fleiss' Kappa as a measure of inter-rater reliability (IRR) across 4 raters.
Results:
The IRR for scoring the overall confidence in the SRs with AMSTAR 2 and the overall domain in ROBIS was fair (AMSTAR 2: κ = 0.30, 95% [confidence interval] CI: 0.17 to 0.43; ROBIS: κ = 0.28, 95% CI: 0.13 to 0.42). AMSTAR 2 confidence in review ratings strongly correlated with the overall domain rating in ROBIS (Spearman rs = 0.84). Mean time for scoring AMSTAR 2 was slightly higher than for ROBIS (18 vs. 16 min), with huge differences between the reviewers.
Conclusion:
Both AMSTAR 2 and ROBIS can be applied to SRs including both randomized controlled trials (RCTs) and non-RCTs. Measurement properties of ROBIS seemed not to be much different when comparing with other studies that include only SRs of RCTs.
Related Concept Videos
Random and Systematic Errors
Minor Losses in Pipes
Valves play a significant role in generating minor losses by obstructing or redirecting the fluid flow. When a valve is closed or partially closed, it restricts the flow...
Bioequivalence Experimental Study Designs: Completely Randomized and Randomized Block Designs
Review and Preview
Percentiles are a type of fractile that partition data into...
Review and Preview
Random Error

