Leveraging Large Language Models with Retrieval-Augmented Generation for Semantic Mapping of Clinical Data Lakes to...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Natural Language Processing for Clinical Data

Background:

• Manual mapping of local clinical concepts to standardized terminologies like SNOMED CT is crucial for semantic interoperability and research but is time-consuming and difficult to scale.
• Existing methods often struggle with the complexity and volume of clinical data found in hospital datalakes.

Purpose of the Study:

• To evaluate a hybrid approach combining retrieval-augmented generation (RAG) and a large language model (LLM) for automated mapping of hospital datalake concepts to SNOMED CT.
• To assess the performance of this AI-assisted method against retrieval-only baselines and determine its potential to reduce expert workload.

Main Methods:

• Developed a pipeline using retrieval-augmented generation (RAG) with embedding-based candidate retrieval followed by LLM-based disambiguation (GPT-4.1-mini).
• Evaluated the pipeline on a dataset of 2,768 concepts from five semantic categories (organisms, healthcare locations, laboratory samples, allergies, clinical assessments) annotated by SIMED.
• Compared the hybrid approach's accuracy (strict, weighted, top-3) against a retrieval-only baseline.

Main Results:

• The RAG-LLM hybrid approach demonstrated superior performance compared to retrieval-only methods across all categories.
• High strict accuracy was achieved for organism concepts (92.8%) and allergy concepts (84.7%).
• Overall strict accuracy was 72.8%, weighted accuracy 76.1%, and top-3 accuracy 83.1%, indicating significant potential despite challenges with certain concept types like healthcare locations and laboratory samples.

Conclusions:

• Combining RAG with LLM-based reasoning offers a promising solution for scalable, AI-assisted semantic interoperability in healthcare.
• The proposed method can effectively reduce the workload of clinical experts while maintaining high accuracy in concept mapping.
• This advancement supports improved semantic consistency in clinical data lakes, facilitating interoperable access to patient-level health data.