CMedRAGBot: A Chinese Medical Chatbot Based on Graph RAG and Large Language Models

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Natural Language Processing

Background:

• Traditional Large Language Models (LLMs) face challenges in clinical medical question-answering (QA), including hallucinations and knowledge update difficulties.
• Knowledge-intensive medical QA tasks require robust systems that can provide accurate and up-to-date information.
• Existing systems struggle with the nuances of Chinese medical terminology and query complexity.

Purpose of the Study:

• To develop a Chinese clinical medical QA system, CMedRAGBot, that addresses the limitations of traditional LLMs.
• To integrate a medical knowledge graph and retrieval-augmented generation (RAG) for enhanced QA performance.
• To improve the accuracy and reliability of answers in Chinese clinical medical question-answering.

Main Methods:

• Construction of a comprehensive Chinese medical knowledge graph covering diseases, medications, and symptoms.
• Development of a Named Entity Recognition (NER) model using Chinese-RoBERTa and BiGRU with data augmentation.
• Implementation of prompt engineering for user query intent recognition and integration of all modules into the CMedRAGBot system.

Main Results:

• CMedRAGBot integration significantly improved the test accuracy of five state-of-the-art LLMs by up to approximately 10%.
• Data augmentation enhanced the NER model's F1 score from 95.27% to 97.55%.
• The intent recognition module improved the understanding of complex queries, further boosting answer accuracy.

Conclusions:

• CMedRAGBot effectively enhances Chinese clinical medical QA systems by mitigating LLM limitations.
• The integration of RAG and medical knowledge graphs is crucial for accurate and reliable medical information retrieval.
• The developed system offers a promising solution for knowledge-intensive clinical medical question-answering tasks.