Heterogeneity of systematic reviews in oncology

Systematic reviews synthesize data across multiple studies to answer a research question, and an important component of the review process is to evaluate the heterogeneity of primary studies considered for inclusion. Little is known, however, about the ways that systematic reviewers evaluate heterogeneity, especially in clinical specialties like oncology. We examined a sample of systematic reviews from this body of literature to determine how meta-analysts assessed and reported heterogeneity. A PubMed search of 6 oncology journals was conducted to locate systematic reviews and meta-analyses. Two coders then independently evaluated the manuscripts for 10 different elements based on an abstraction manual. The initial PubMed search yielded 337 systematic reviews from 6 journals. Screening for exclusion criteria (nonsystematic reviews, genetic studies, individual patient data, etc.) found 155 articles that did not meet the definition of a systematic review. This left a final sample of 182 systematic reviews across 4 journals. Of these reviews, 50% (91/182) used varying combinations of heterogeneity tests, and of those, 16% (15/91) of review authors noted excessive heterogeneity and opted to not perform a meta-analysis. Of the studies that measured heterogeneity, 51% (46/91) used a random-effects model, 7% (8/91) used a fixed-effects model, and 43% (39/91) used both. We conclude that use of quantitative and qualitative heterogeneity measurement tools are underused in the 4 oncology journals evaluated. Such assessments should be routinely applied in meta-analyses.