Evaluation of a document search engine in a clinical department system

MorphoSaurus, a concept-based document search engine,was incorporated into an EHR system in order to support search across the whole corpus of patient discharge letters and other clinically relevant documents. A user survey showed a general satisfaction with the system and revealed novel usages for information stored in discharge letters.The retrieval system was also used to identify relevant documents for a five-year retrospective survey of suspicious syphilis cases in the department. This retrieval scenario was used to assess the performance of MorphoSaurus against a manually created gold standard. A substring search for the German words "syphilis" and"lues" was used as baseline. The system yielded a precision p = 20.1% and a recall r = 100%. The values for the substring "syphilis" were p = 65.5% and r = 47.5%, for"lues" p = 15.4% and r = 87.7%. The results support the use of the proposed recall-oriented search across EHR documents to acquire valid and complete data for epidemiology studies in hospital populations.