Combining image features, case descriptions and UMLS concepts to improve retrieval of medical images

This paper evaluates a system, UBMedTIRS, for retrieval of medical images. The system uses a combination of image and text features as well as mapping of free text to UMLS concepts. UBMedTIRS combines three publicly available tools: a content-based image retrieval system (GIFT), a text retrieval system (SMART), and a tool for mapping free text to UMLS concepts (MetaMap). The system is evaluated using the ImageCLEFmed 2005 collection that contains approximately 50,000 medical images with associated text descriptions in English, French and German. Our experimental results indicate that the proposed approach yields significant improvements in retrieval performance. Our system performs 156% above the GIFT system and 42% above the text retrieval system.