Errors in search strategies were identified by type and frequency.

Area of Science:

• Medical Informatics
• Evidence-Based Medicine
• Systematic Review Methodology

Background:

• Electronic search strategies are crucial for systematic reviews.
• Errors in these strategies can compromise the reliability of the evidence base.
• The Cochrane Collaboration is a leading producer of systematic reviews.

Purpose of the Study:

• To determine the frequency and types of errors in electronic search strategies used in systematic reviews.
• To assess the impact of these errors on the integrity of the evidence base.

Main Methods:

• Analysis of MEDLINE search strategies from systematic reviews in the Cochrane Library (Issue 3, 2002).
• Inclusion criteria: randomized or quasi-randomized controlled trials, reported studies, use of Cochrane Highly Sensitive Search Strategy components.
• Exclusion criteria: inadequate reporting or duplicate strategies.
• Two librarians assessed strategies for 11 potential error types using dual review with consensus.

Main Results:

• 63 out of 105 MEDLINE search strategies were assessed; 31 were excluded for inadequate reporting, 11 for duplication.
• 90.5% of assessed strategies contained at least one error (median 2 errors).
• Errors potentially lowering recall were found in 82.5% of strategies.
• Common errors included missed MeSH terms (44.4%), unwarranted MeSH term explosion (38.1%), and irrelevant terms (28.6%).

Conclusions:

• Systematic reviews, even those from reputable sources like the Cochrane Collaboration, commonly exhibit errors in their electronic search strategies when sufficiently reported.
• These errors can impact the completeness and accuracy of the evidence identified.
• Implementing additional peer review steps is essential to enhance search quality and minimize errors in systematic reviews.