Augmenting large language models with clinical knowledge graph for personalized perioperative fluid therapy question...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support Systems

Background:

• Personalized perioperative fluid therapy is crucial for minimizing postoperative complications.
• Large Language Models (LLMs) face challenges in healthcare, including hallucinations and lack of domain-specific knowledge, hindering their use in fluid therapy.
• Retrieval-Augmented Generation (RAG) and Knowledge Graphs (KGs) offer potential solutions for improving LLM accuracy and reliability.

Purpose of the Study:

• To develop and evaluate a domain-adapted retrieval strategy for personalized perioperative fluid therapy question answering.
• To construct a Personalized Fluid Therapy Knowledge Graph (PFTKG) and adapt a graph-based RAG strategy (GraphRAG).
• To compare the performance of GraphRAG against traditional RAG (DocRAG) and prompting strategies (Vanilla, CoT, RoT) across multiple LLMs.

Main Methods:

• Construction of a Personalized Fluid Therapy Knowledge Graph (PFTKG) with 6,490 entities and 15,687 relationships.
• Adaptation of GraphRAG, a graph-based RAG strategy utilizing community detection and recursive summarization for finding-level retrieval.
• Evaluation using knowledge-based (300 questions) and retrospective case-based (262 questions from 206 patients) datasets, assessing accuracy, honesty, and error composition across GPT-4o, Claude Opus 4, and Gemini 2.5 Pro.

Main Results:

• GraphRAG achieved the highest average accuracy on the knowledge-based question set (96.89% multiple-choice, 66.44% open-ended).
• On the retrospective case-based set, GraphRAG demonstrated superior performance (71.12% accuracy) compared to DocRAG (62.47%) and prompting strategies (52-54%).
• GraphRAG reduced context-irrelevant errors and improved acknowledgment of uncertainty when a "Don't know" option was included.

Conclusions:

• GraphRAG is an effective domain-adapted retrieval strategy for personalized perioperative fluid therapy question answering.
• Integrating a clinical knowledge graph with hierarchical summarization and finding-level retrieval enhances answer accuracy and promotes conservative responses under uncertainty.
• The findings support the potential of GraphRAG for future clinically integrated studies in perioperative care.