Value and usability of unpublished data sources for systematic reviews and network meta-analyses

Peer-reviewed publications and conference proceedings are the mainstay of data sources for systematic reviews and network meta-analyses (NMA), but access to informative unpublished data is now becoming commonplace. To explore the usefulness of three types of 'grey' literature-clinical trials registries, clinical study reports and data from regulatory authorities-we conducted four case studies. The reporting of outcome data in peer-reviewed publications, the clinical trials registries and the clinical study reports for two clinical trials-one in melanoma, one in juvenile idiopathic arthritis (JIA)-was examined. In addition, we assessed the value of including unpublished data from the European Medicines Agency (EMA) and US Food and Drug Administration (FDA) in evidence syntheses of hepatitis C virus (HCV) and chronic obstructive pulmonary disease (COPD), respectively. For the clinical trials in melanoma and JIA, we identified outcome parameters on ClinicalTrials.gov additional to those reported in the peer-reviewed publications: subgroup data and additional efficacy end points/extended follow-up, respectively. The clinical study report also provided results for several subgroups unavailable elsewhere. For HCV and COPD, additional outcome data were obtained from the EMA European Public Assessment Report (EPAR) and the FDA, respectively, including data on subgroups and mortality. We conclude that data from these grey literature sources have the potential to influence results of systematic reviews and NMAs, and may thus have implications for healthcare decisions. However, it is important to consider carefully the availability, reliability and consequent usability of these data sources in systematic reviews and NMAs.