Evaluating the UMLS as a source of lexical knowledge for medical language processing

Area of Science:

• Natural Language Processing
• Medical Informatics

Background:

• Medical Language Processing (MLP) systems require specialized lexicons for accurate medical terminology recognition.
• Lexicon acquisition is costly and time-intensive, hindering MLP system development.
• The Unified Medical Language System (UMLS) offers a valuable resource for acquiring medical lexical knowledge.

Purpose of the Study:

• To describe methods for automatically creating lexical entries using the UMLS.
• To generate two lexicons for medical language processing.
• To evaluate the performance of an MLP system (MedLEE) using these new lexicons.

Main Methods:

• Developed methods to automatically generate lexical entries from the UMLS.
• Created a primary lexicon using the UMLS.
• Generated a supplemental lexicon by augmenting an existing MedLEE lexicon with UMLS entries.
• Evaluated MedLEE performance with the UMLS-based lexicon, the supplemental lexicon, and the original MedLEE lexicon.

Main Results:

• The lexicon primarily based on UMLS achieved .86 accuracy, .60 sensitivity, and .96 specificity.
• The original MedLEE lexicon alone achieved .93 accuracy, .81 sensitivity, and .93 specificity, significantly outperforming the UMLS-based lexicon except in specificity.
• Supplementing the MedLEE lexicon with UMLS entries resulted in performance identical to the original MedLEE lexicon.

Conclusions:

• While the UMLS is a rich resource, a lexicon derived solely from it did not outperform an existing specialized lexicon (MedLEE).
• Supplementing an established lexicon with UMLS data did not improve performance over the original lexicon.
• The quality and coverage of existing specialized lexicons remain critical for optimal MLP system performance.