Reporting methodological search filter performance comparisons: a literature review

Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.

Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.

Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.

Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.