Biomedical knowledge graph-optimized prompt generation for large language models

Area of Science:

• Biomedical Informatics
• Artificial Intelligence

Background:

• Large language models (LLMs) face challenges in knowledge-intensive domains like biomedicine.
• Current solutions like pretraining and fine-tuning incur high computational costs and require domain expertise.

Purpose of the Study:

• To introduce a token-optimized Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework.
• To leverage the SPOKE biomedical knowledge graph (KG) with LLMs for generating accurate biomedical text.

Main Methods:

• Developed a KG-RAG framework utilizing the SPOKE KG and LLMs (Llama-2, GPT-3.5, GPT-4).
• Implemented context extraction with minimal graph schema and context pruning via embedding methods.
• Optimized token consumption for cost-effective RAG on proprietary LLMs.

Main Results:

• Achieved over 50% reduction in token consumption without compromising accuracy.
• Enhanced LLM performance across diverse biomedical prompts, providing provenance and statistical evidence.
• Boosted Llama-2 model performance by 71% on a challenging multiple-choice question dataset.

Conclusions:

• The KG-RAG framework effectively combines explicit KG knowledge and implicit LLM knowledge in a token-optimized manner.
• This approach enhances the adaptability of general-purpose LLMs for domain-specific biomedical questions cost-effectively.
• The framework empowers open-source models for specialized tasks and improves proprietary LLM performance.