A comparison of yield in a non-systematic and a systematic review

To compare the results of a non-systematic (A) and a systematic review (B) of test performance in oral cancer and precancer screening in terms of literature coverage, and qualitative and quantitative information yield.

For A, two databases were searched using basic search terms. Review B followed published guidelines. Nine databases were searched using three complex facets. In both reviews, one reviewer made the initial selection of relevant studies and, in A, carried out full text screening. In B, two independent reviewers carried out full text screening and then conferred on a final list of studies for inclusion and meta-analysis.

In search A, 60 papers, and in B, 481 papers, were retrieved. Thirty from A, and 47 from B were selected for full text screening. From A, six papers, and from B, eight papers (six selected by the first reviewer and eight by the second; kappa = 0.83), were designated for inclusion in meta-analysis. The six papers from A were common to all selections. Seven papers were finally included in B. Compared with the non-systematic review, meta-analysis in the systematic review reduced the 95% CI for sensitivity from 0.594, 0.912 (= 0.318) to 0.713, 0.919 (= 0.189) resulting from the inclusion of a larger number of verified cases. The 95% CI for specificity changed only from 0.941, 0.991 (= 0.050) to 0.930, 0.982 (= 0.052).

Both reviews had the same exacting and specific inclusion criteria. The papers ultimately included in each were closely matched. Advantages of the systematic review were 1) it afforded a degree of confidence that the literature had been exhaustively and systematically searched, 2) the use of a formal data extraction process ensured that clinical heterogeneity among the studies and its consequences were thoroughly explored, 3) the increased number of verified cases included in the systematic review meta-analysis led to the pooled estimate for sensitivity having greater precision.