Med-KAG: Preliminary Results of a Medical Knowledge-Augmented Generation Approach

Area of Science:

• Artificial Intelligence in Medicine
• Biomedical Informatics
• Clinical Decision Support Systems

Background:

• Clinical decision support systems (CDSS) often struggle with accuracy and transparency.
• Large language models (LLMs) show promise but are prone to generating incorrect information (hallucinations).
• Integrating structured medical knowledge can enhance the reliability of AI in healthcare.

Purpose of the Study:

• To propose Med-KAG, a novel AI assistant architecture for clinical decision support.
• To extend the Retrieval-Augmented Generation (RAG) paradigm with a medical knowledge graph.
• To improve the accuracy, transparency, and reduce hallucinations in AI-driven clinical assistance.

Main Methods:

• Developed Med-KAG, an AI architecture integrating a knowledge graph from the Unified Medical Language System (UMLS) Metathesaurus with the RAG paradigm.
• Grounded AI responses in verified biomedical relationships between diseases, symptoms, and treatments.
• Evaluated a knowledge-enhanced LLM variant against a baseline model using the MedQA-US dataset.

Main Results:

• The knowledge-enhanced LLM demonstrated comparable accuracy to the baseline model.
• The study identified the retriever component as the primary source of error.
• Preliminary evaluation highlights the potential of structured knowledge integration.

Conclusions:

• Med-KAG architecture shows promise for more reliable and explainable clinical decision support.
• Combining structured medical knowledge with generative AI can mitigate LLM limitations.
• Further research is needed to address limitations, particularly within the retrieval component.