Systematic literature searching in policy relevant, inter-disciplinary reviews: an example from culture and sport

Within the systematic review process, the searching phase is critical to the final synthesis product, its use and value. Yet, relatively little is known about the utility of different search strategies for reviews of complex, inter-disciplinary evidence. This article used a recently completed programme of work on cultural and sporting engagement to conduct an empirical evaluation of a comprehensive search strategy. Ten different types of search source were evaluated, according to three dimensions: (i) effectiveness in identifying relevant studies; (ii) efficiency in identifying studies; and (iii) adding value by locating studies that were not identified by any other sources. The study found that general bibliographic databases and specialist databases ranked the highest on all three dimensions. Overall, websites and journals were the next most valuable types of source. For reviewers, these findings highlight that general and specialist databases should remain a core component of the comprehensive search strategy, supplemented with other types of sources that can efficiently identify unique or grey literature. For policy makers and other research commissioners, this study highlights the value of methodological analysis for improving the understanding of, and practice in, policy relevant, inter-disciplinary systematic reviews. Copyright © 2011 John Wiley & Sons, Ltd.