Methodological approaches for assessing certainty of the evidence in umbrella reviews: A scoping review

The number of umbrella reviews (URs) that compiled systematic reviews and meta-analysis (SR-MAs) has increased dramatically over recent years. No formal guidance for assessing the certainty of evidence in URs of meta-analyses exists nowadays. URs of non-interventional studies help establish evidence linking exposure to certain health outcomes in a population. This study aims to identify and describe the methodological approaches for assessing the certainty of the evidence in published URs of non-interventions.

We searched from 3 databases including PubMed, Embase, and The Cochrane Library from May 2010 to September 2021. We included URs that included SR-MAs of studies with non-interventions. Two independent reviewers screened and extracted data. We compared URs characteristics stratified by publication year, journal ranking, journal impact factor using Chi-square test.

Ninety-nine URs have been included. Most were SR-MAs of observational studies evaluating association of non-modifiable risk factors with some outcomes. Only half (56.6%) of the included URs assessed the certainty of the evidence. The most frequently used criteria is credibility assessment (80.4%), followed by GRADE approach (14.3%). URs published in journals with higher journal impact factor assessed certainty of evidence than URs published in lower impact group (77.1 versus 37.2% respectively, p < 0.05). However, criteria for credibility assessment used in four of the seven URs that were published in top ranking journals were slightly varied.

Half of URs of MAs of non-interventional studies have assessed the certainty of the evidence, in which criteria for credibility assessment was the commonly used method. Guidance and standards are required to ensure the methodological rigor and consistency of certainty of evidence assessment for URs.