Compliance with existing standards: overview comparing Cochrane with non-Cochrane living systematic reviews

Living systematic reviews (LSRs) have been published with increasing frequency since the start of the COVID-19 pandemic, but there has been little formal evaluation of their methodological consistency and compliance with best-practice standards. The objective of this overview was to evaluate the extent and nature of current LSRs in health research, as well as the degree to which they acknowledge and adhere to existing methodological standards.

We searched MEDLINE, Embase, Cochrane Database of Systematic Reviews, PROSPERO, Scopus, Epistemonikos, and TRIPdatabase from inception to 24 June 2022. We ran a restricted update search for LSRs published 1 January 2025 to 9 January 2026 to see if conclusions would be similar. We extracted data in duplicate. The primary outcome was the number and proportion of LSRs that conform to existing living standards: criteria and guidance for their need and conduct as published by Cochrane. Secondary outcomes assessed reliability and other reporting aspects. Outcomes were assessed according to explicit author reporting for each review.

Final analysis included all 23 Cochrane LSRs at the time and a randomly selected 23 non-Cochrane LSRs (among a total of 113). Ten (43%) Cochrane LSRs clearly met the three need criteria for living mode. None of the non-Cochrane LSRs met all three criteria. Cochrane LSRs were also more likely than non-Cochrane LSRs to adhere to Cochrane methodological guidance for the conduct of LSRs, with six (26%) of 23 Cochrane reviews meeting all guidance items. None of the non-Cochrane reviews met all guidance items. In the restricted update (10 LSRs included: five Cochrane and five non-Cochrane), Cochrane LSRs were still more likely to adhere to the three need criteria (four of five, 80%) and five methodological guidance items (three of five, 60%) compared to non-Cochrane LSRs (one of five, or 20% for criteria and guidance adherence). Ten (22%) of all 46 included LSRs were cited by clinical practice guidelines, according to PubMed.

This overview highlights a lack of methodological consistency among LSRs, potentially stemming from a lack of awareness of Cochrane's LSR standards. One criterion to become living is that the topic is a priority for decision-making; however, there is little evidence that LSRs are cited frequently in clinical practice guidelines. Establishing clear, objective LSR standards would help to support both Cochrane and non-Cochrane authors and improve evidence-based health care.

PROSPERO CRD42022332327.