CUI-Curate: a GraphRAG-based framework for automated clinical concept curation for NLP applications.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Knowledge Representation

Background:

• Clinical named entity recognition tools map text to Unified Medical Language System (UMLS) Concept Unique Identifiers (CUIs).
• Downstream tasks often require concept sets, not single CUIs, which are labor-intensive to create.
• Existing tools poorly support the construction of comprehensive UMLS concept sets.

Purpose of the Study:

• To present CUI-Curate, a novel graph-based retrieval-augmented-generation (GraphRAG) framework for automated UMLS concept set curation.
• To enable scalable, reproducible, and cost-efficient generation of clinician-reviewable concept sets.
• To enhance clinical natural language processing (NLP) and phenotyping applications.

Main Methods:

• Constructed and embedded a UMLS knowledge graph for semantic retrieval.
• Utilized graph-based expansion to retrieve candidate CUIs.
• Filtered and classified candidate CUIs using large language models (GPT-5 and Qwen3-32B).

Main Results:

• CUI-Curate generated substantially larger and more complete concept sets compared to manual benchmarks.
• GPT-5 and Qwen3-32B models demonstrated high performance in classifying CUIs, with GPT-5 outperforming manual curation.
• The framework achieved high recall of definitive concepts with manageable candidate sets and proved inexpensive and stable.

Conclusions:

• CUI-Curate provides a scalable and cost-efficient method for generating UMLS concept sets.
• The framework is suitable for clinical NLP and phenotyping applications, offering clinician-reviewable outputs.
• Automated concept set curation using GraphRAG significantly improves upon traditional methods.