Evaluation of tools used to assess adherence to PRISMA 2020 reveals inconsistent methods and poor tool...

Numerous studies have assessed the adherence of published systematic reviews to the PRISMA 2020 statement. We aimed to summarize the characteristics and methods of development of the tools used to assess adherence in these studies.

MEDLINE, Embase, and PsycINFO (all via Ovid) were searched on January 20, 2025, to locate studies that assessed adherence of systematic reviews of health interventions to the PRISMA 2020 statement. Two authors independently screened all records and extracted data. We examined three aspects of the tools used to assess adherence to PRISMA 2020: i) characteristics of the assessment tool, ii) methods used to develop and validate the tool, and iii) processes used to apply the tool. We classified a tool as "implementable" by researchers external to the tool developers if authors reported the exact wording of each item, its response options, guidance on how to operationalize all response options, and the algorithms used to aggregate judgments and quantify adherence.

We included 24 meta-research studies that had assessed adherence to PRISMA 2020 in 2766 systematic reviews published between 1989 and 2024. Most authors assessed adherence to PRISMA 2020 in its entirety (N = 15/24, 63%), with the remaining nine (37%) assessing adherence to one or a subset of domains (eg, abstract, search methods, and risk of bias assessment methods). Psychometric testing of the assessment tool was reported by five studies (21%), all of which assessed the inter-rater reliability of the tool. Only one (4%) reported how response options for all items were operationalized. According to our criteria, only one assessment tool was classified as implementable (N = 1/24, 4%). No authorship team used the same methods to assess adherence to the PRISMA 2020 statement. However, information on some tool characteristics was unavailable for several studies.

Our findings demonstrate variation and inadequacies in the methods and reporting of tools used to assess adherence to the PRISMA 2020 statement. We have commenced work on a standardized PRISMA 2020 assessment tool to facilitate accurate and consistent assessments of adherence of systematic reviews to PRISMA. In the interim, we provide some recommendations for how meta-researchers interested in assessing adherence of systematic reviews to the PRISMA 2020 statement can transparently report the findings of their assessments.