In search of controlled evidence for health care quality improvement

The purpose of this study was to measure the efficiency of simple searches in retrieving controlled evidence about specific primary health care quality improvement interventions and their effects. Searches were conducted to retrieve evidence on seven interventions and seven effect variables. Specific words and the closest Medical Subject Headings (MeSH) recommended by professional librarians were used to search the MEDLINE database. Searches were restricted to the MeSH publication type "randomized controlled trial." Two reviewers independently judged retrieved citations for relevancy to the selected interventions and effects. In selecting MeSH terms, the average agreement among librarians was 64.3% (+/-26.1) for interventions and 57.1% (+/-19.9) for effects. Analysis of the 755 retrieved reports showed that MeSH term searches had an overall recall rate of 58% while the same rate for textword searches was significantly lower (11%, p < .001). The difference in overall precision rates was nonsignificant (26% versus 33%, p = .15). In the group of MeSH searches, overall precision and recall was significantly lower for effects than for interventions (12% versus 52%, p < .001 and 41% versus 69%, p < .001). Two textwords appeared in more than 25% of the benchmark collection: reminder (25.7%) and cost (25.0%). The results of this study indicate that information needs for health care quality improvement cannot be met by simple literature searches. Certain MeSH terms and combinations of textwords yield moderately efficient recall and precision in literature searches for health care quality improvement. Clinicians and physician executives gaining direct access to bibliographic database could probably be better served by structured indexing of critical aspects of randomized controlled clinical trials: design, sample, interventions, and effects.