A systematic survey showed important limitations in the methods for assessing drug safety among systematic reviews

This study aimed to examine the design, conduct, and analysis of systematic reviews assessing drug safety through a cross-sectional survey.

We searched PubMed to identity systematic reviews published in the Cochrane Database of Systematic Reviews and Core Clinical Journals indexed in 2015 and randomly sampled systematic reviews assessing drug effects at a 1:1 ratio of Cochrane and non-Cochrane reviews. Teams of two investigators independently conducted study screening and collected data, using prespecified, standardized questionnaires. In addition to general information, we collected details about the planning and analyses of safety outcomes.

We included 120 systematic reviews, including 60 Cochrane and 60 non-Cochrane reviews. Most reviews searched PubMed/MEDLINE (n = 117, 97.5%), EMBASE (n = 105, 87.5%), and Cochrane CENTRAL (n = 110, 91.7%) and conducted independent and duplicate study selection (n = 98, 81.7%), risk of bias assessment (n = 105, 87.5%), and data collection (n = 105, 87.5%). Only nine (7.5%) reviews clearly defined safety outcomes, and seven (5.8%) defined a primary safety outcome; none stated whether the primary safety outcome was predefined. Among the 80 reviews that pooled the primary dichotomous safety data across studies, less than half (41%, n = 33) conducted subgroup analysis to explore for sources of heterogeneity or reported a GRADE assessment for the overall quality of evidence. Cochrane reviews were more likely to provide a study protocol (100% vs. 23.3%; P < 0.001), involve methodologists (53.3% vs. 20.0%; P < 0.001), and report a GRADE assessment for the primary safety outcome (70.6% vs. 19.6%; P < 0.001).

Our findings highlighted areas for improved planning and analysis in the assessment of drug safety among systematic reviews. Cochrane reviews were superior to non-Cochrane reviews; however, most reviews did not prespecify their safety outcomes or methods for analysis, explore sources of heterogeneity among pooled effects, or assess the overall quality of evidence with the GRADE approach.