Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data

The aim of the study was to assess the completeness of reporting of methods in overviews.

Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented.

We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7-32) systematic reviews (SRs), 22% reported working from a protocol, 36% reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34% reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56% framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44% reported eligibility criteria based on PICO, and 76% reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20%) or statistical synthesis (26%). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30%], discrepant or missing data [14%], and discordant results/conclusions across reviews [20%]).

Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.