Evidence-based retrieval in evidence-based medicine

Clinical decisions based on a meta-analysis that is based on an ineffective retrieval strategy may have serious negative consequences for patients. The study objective was to investigate the extent to which meta-analyses report proof of their retrieval strategies' effectiveness.

The authors examined a random sample (n = 100) of articles in the 1996 to 2002 full-text subset of Ovid MEDLINE indexed as "meta-analysis." We classified the articles in three ways: the article (A) reported both a retrieval strategy in sufficient detail (such that it could be repeated) and with evidence of the strategy's effectiveness, (B) reported a retrieval strategy in sufficient detail but not with evidence of the strategy's effectiveness, or (C) neither reported a strategy in detail nor evidence of the strategy's effectiveness. Articles classified as (A) were further classified according to the level of evidence reported.

Of the eighty-nine articles in our final analysis, six (6.7%) were classified as category (A), fifty-seven (64%) as (B), and twenty-six (29%) as (C). Articles in category (A) reported a previously validated search, a published strategy, or strategy based on expert opinion.

Peer-review standards must be developed that require authors of meta-analyses to report evidence for the effectiveness of their retrieval strategies.