Pitfalls in meta-analyses on adverse events reported from clinical trials

In recent years, comparative effectiveness research has been more aggressively pursued as a means to improve health care, including systematic reviews and meta-analyses to inform health policy decision making. Because most clinical trials have pre-specified approaches to collecting data on efficacy, the value of systematic reviews and meta-analyses in assessing efficacy outcomes is generally accepted. In contrast, collection of data on adverse events is seldom well structured. Hence, the methodological considerations for comparing adverse events from such non-aligned sources differ substantially from those for comparing efficacy endpoints. We address several important pitfalls in performing systematic reviews and meta-analyses on adverse events in clinical trials, and we offer recommendations for remedies. Some pitfalls arise from the fact that adverse events often are not the primary endpoints in clinical trials, hence incomplete reporting, inconsistent event definitions, various level of effort in reporting unexpected adverse events, and inappropriate use of statistical testing. Others are posed by certain important characteristics of adverse events data. The very concept of "adverse events" may skew the ascertainment, attribution, and reporting of the events. In addition, problems for meta-analysis methods arise in situations involving zero or rare events and withdrawal or loss to follow-up because of adverse events. We highlight recent initiatives that may improve the assessment and cross-study summary of adverse events. We anticipate that future guidance for conducting systematic reviews and meta-analyses will evolve to address the important methodological pitfalls we highlight here, and the practice of assessing the totality of evidence on drug safety will be improved.