Methodological use and report of AMSTAR 2: a cross-sectional study

Area of Science:

• Meta-research
• Evidence synthesis methodology
• Systematic review quality assessment

Background:

• The Appraisal Tool for Systematic Reviews 2 (AMSTAR 2) is a widely used tool for assessing the methodological quality of systematic reviews.
• Understanding the reporting and methodological practices associated with AMSTAR 2 is crucial for ensuring the reliability and reproducibility of evidence syntheses.
• Previous research has highlighted variability in the application and reporting of AMSTAR 2.

Purpose of the Study:

• To analyze the reporting and methodology of studies that used AMSTAR 2 for assessing systematic reviews.
• To evaluate temporal trends in AMSTAR 2-related reporting practices between 2018 and 2023.

Main Methods:

• A cross-sectional meta-research study was conducted, analyzing 444 scholarly articles that applied AMSTAR 2.
• Data on reporting and methodology were extracted and analyzed.
• Temporal trends in seven AMSTAR 2 reporting practices were evaluated using correlation tests and linear regression.

Main Results:

• Most AMSTAR 2 assessments (78%) involved multiple authors, with 91% conducted independently.
• The recommended rating system (84%) and critical domains (82%) were frequently used.
• While confidence ratings were reported in 76% of studies, only 30% provided a rationale.
• Domain-level judgments were presented in 72% of studies, but with justifications in only 3%.
• Significant increasing trends were observed in most reporting practices over time, with 1-20 additional studies per year adopting recommended procedures.

Conclusions:

• Authors should enhance transparency by clearly defining key domains and explaining the determination of overall confidence ratings.
• Reporting both overall ratings and domain-level judgments with justifications is essential for replication and updating of evidence syntheses.
• Despite improvements in adherence to AMSTAR 2 guidance, substantial gaps in justification and transparency require attention.