A comparison of retrieval effectiveness for three methods of indexing medical literature

Conventional approaches to indexing medical literature include the human assignment of terms from a controlled vocabulary, such as MeSH, or the computer assignment of all words in the title and abstract as indexing terms. Human indexing suffers from inconsistency, while word-based indexing suffers from the multiple meanings of words. SAPHIRE is a computer program designed to provide indexing using controlled terms that are assigned by computer, based on their occurrence in the title and abstract. In this first evaluation of SAPHIRE, the authors compared the retrieval performance of the three indexing approaches--human-based MEDLINE with text words; machine-based SAPHIRE with text words; and text words only--for searches by measuring recall and precision for each search using a test collection of 200 abstracts. The abstracts were judged by human reviewers for relevance as applied to 12 literature queries. The results suggest that text word indexing is more effective than indexing with MeSH terms. SAPHIRE's indexing performance was slightly inferior but the program has other advantageous features.