Related Experiment Video
Updated: Jan 24, 2026

Achieving Moderate Pressures in Sealed Vessels Using Dry Ice As a Solid CO2 Source
Published on: August 17, 2018
A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool
Robert C Lorenz1, Katja Matthias2, Dawid Pieper3
1Federal Joint Committee (Healthcare), Medical Consultancy Department, Gutenbergstr. 13, 10587 Berlin, Germany; Division of Social and Preventive Medicine, University of Potsdam, Research Focus Cognitive Sciences, Am Neuen Palais 10, Potsdam 14469, Germany.
Objectives:
The objectives of this study were to determine the interrater reliability (IRR) of assessment of multiple systematic reviews (AMSTAR) 2 for reviews of pharmacological or psychological interventions for the treatment of major depression, to compare it to that of AMSTAR and risk of bias in systematic reviews (ROBIS), and to assess the convergent validity between the appraisal tools.
Study Design And Setting:
Two groups of four raters were each assigned one of two samples of 30 systematic reviews. All eight raters applied AMSTAR 2 to their sample. Each group also applied either AMSTAR or ROBIS. Fleiss' kappa and Gwet's AC1 were calculated, and agreement between the tools was assessed.
Results:
The median kappa values as a measure of IRR indicated a moderate agreement for AMSTAR 2 (median = 0.51), a substantial agreement for AMSTAR (median = 0.62), and a fair agreement for ROBIS (median = 0.27). Validity results showed a positive association for AMSTAR and AMSTAR 2 (r = 0.91) as well as ROBIS and AMSTAR 2 (r = 0.84). For the overall rating, AMSTAR 2 showed a high concordance with ROBIS and a lower concordance with AMSTAR.
Conclusion:
The IRR of AMSTAR 2 was found to be slightly lower than the IRR of AMSTAR and higher than the IRR of ROBIS. Validity measurements indicate that AMSTAR 2 is closely related to both ROBIS and AMSTAR.
More Related Videos
10:39Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
08:51Utilizing the Ethylene-releasing Compound, 2-Chloroethylphosphonic Acid, as a Tool to Study Ethylene Response in Bacteria
Published on: November 10, 2016
Related Concept Videos
Reliability and Validity
The Sense of Self: Reflected Self-Appraisal and Social Comparison
Lazarus's Cognitive Appraisal Theory
Primary Appraisal:...
Distribution Reliability and Automation
Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
Data Validation
Key parameters for method validation include: