Two complementary AI approaches for predicting UMLS semantic group assignment: heuristic reasoning and deep learning.

Area of Science:

• Medical Informatics
• Artificial Intelligence
• Natural Language Processing

Background:

• The Unified Medical Language System (UMLS) Metathesaurus is a crucial resource for integrating biomedical vocabularies.
• Assigning new atoms to semantic groups (SGs) is a time-consuming manual process.
• Accurate SG prediction can streamline the integration of new terms into the UMLS.

Purpose of the Study:

• To develop and evaluate heuristic, deep learning (DL), and hybrid AI methods for predicting Semantic Group (SG) assignments for new UMLS Metathesaurus atoms.
• To achieve a target accuracy of ≥95% for SG prediction.
• To assess the potential utility of AI-driven SG prediction as an intermediate step in UMLS concept assignment.

Main Methods:

• A heuristic "waterfall" approach using a sequence of 7 prediction methods was implemented.
• A DL approach utilized BioWordVec and SapBERT embeddings fed into a multilayer neural network.
• Hybrid methods were developed by combining heuristic and DL approaches, incorporating probability estimations for accuracy.

Main Results:

• The heuristic waterfall approach achieved 94.3% accuracy on 1,563,692 new unseen atoms.
• The DL approach also reached 94.3% accuracy on the same dataset.
• The hybrid AI methods demonstrated superior performance, achieving an average accuracy of 96.5%.

Conclusions:

• AI methods, particularly hybrid approaches, can accurately predict SG assignments for new UMLS atoms.
• AI-driven SG prediction shows promise as an intermediate step to accelerate the manual assignment of new atoms to UMLS concepts.
• Combining heuristic and DL methods yields better SG prediction results than either method alone.