Mapping Drug Terms via Integration of a Retrieval-Augmented Generation Algorithm with a Large Language Model.

Area of Science:

• Pharmacoinformatics
• Natural Language Processing
• Computational Linguistics

Background:

• Accurate drug name mapping across international vocabularies is crucial for global pharmaceutical data interoperability.
• Traditional methods like string comparison and vector similarity face challenges in handling linguistic variations and complex drug names.

Purpose of the Study:

• To evaluate the efficacy of integrating retrieval-augmented generation (RAG) and large language models (LLMs) for enhanced drug name mapping.
• To compare the performance of RAG-enhanced LLMs against conventional vector similarity techniques.

Main Methods:

• Drug ingredient names were translated from Japanese to English.
• Drug concepts were extracted from OHDSI vocabulary and mapped to RxNorm using vector similarity (BioBERT embeddings) as a baseline.
• Large language models integrated with RAG were developed to refine candidate selection.
• Performance was assessed by comparing RAG-LLM efficacy against baseline vector similarity methods.

Main Results:

• The combined LLM + RAG approach significantly outperformed traditional vector similarity methods.
• Hit rates for Mixtral 8x7b and GPT-3.5 models exceeded 90%, compared to a baseline of 64%.
• R-precision improved from 23% (baseline) to 41%-50% with LLM + RAG, indicating better alignment with human evaluation.

Conclusions:

• Integrating RAG with LLMs offers a superior method for drug name mapping compared to conventional techniques.
• This approach provides a more refined and accurate solution for global drug information mapping.