Enhancing LLM-based clinical reasoning in anesthesiology via graph-augmented retrieval and explainable generation

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support Systems

Background:

• Large language models (LLMs) show promise for clinical decision support but require enhanced analytical reasoning.
• Integrating structured medical knowledge is crucial for improving LLM performance in specialized domains like anesthesiology.

Purpose of the Study:

• To enhance the capabilities of large language models (LLMs) for anesthesiology decision support.
• To develop a graph-based Retrieval-Augmented Generation (RAG) framework for improved analytical reasoning and evidence-based results.

Main Methods:

• Constructed an Anesthesiology Knowledge Graph using UMLS, comprising over 212,000 entities and 529,000 relations.
• Developed AnesGraph-RAG, a graph-based RAG framework with pre-retrieval judgment, hybrid retrieval, and Chain-of-Thought prompting.
• Introduced AnesGraph-RAG-UD, incorporating unfamiliarity-driven retrieval and structural querying for improved token efficiency.

Main Results:

• The Anesthesiology Knowledge Graph achieved high entity recall (82.19%) and precision (82.13%).
• AnesGraph-RAG outperformed ChatGPT-3.5-turbo, ChatGPT-4o, and DeepSeek-V3 on an anesthesiology exam dataset, with accuracy improvements of 2.4%, 7.3%, and 3.7%, respectively.
• Performance gains were significant in professional knowledge and practical skills assessment, with enhanced explanations providing detailed evidence.

Conclusions:

• The graph-based RAG framework significantly improves LLM accuracy and interpretability in anesthesiology decision support.
• Integrating domain-specific knowledge graphs with LLMs offers practical value for clinical decision support systems.
• The study highlights the potential of structured knowledge integration for advancing AI in medicine.