Knowledge Engineering for Medical Vocabularies Using Large Language Models

Medical vocabularies are essential tools for capturing, classifying, and analyzing healthcare data. However, the creation and maintenance of these vocabularies are often labor-intensive and costly. This preliminary study evaluates the feasibility of using large language models (LLMs) to automate three key tasks in medical vocabulary management: term similarity, subsumption, and grouping. Using 1,533 cardiovascular terms from SNOMED CT, we applied GPT-4o and assessed the performance of 3 elementary tasks against OHDSI standardized vocabularies. While LLMs demonstrated high precision across tasks (0.78 for term similarity, 0.74 for term subsumption, 0.78 for term grouping), recall was notably lower (0.41 for term similarity, 0.08 for term subsumption, 0.52 for term grouping), indicating gaps in coverage. Overall, LLMs show promise for medical vocabulary tasks but require further refinement for clinical specificity and completeness. Future work should focus on enhancing recall, reducing hallucinations, and evaluating scalability across broader terminology sets.