Knowledge Graph Augmented Large Language Models for Disease Prediction

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Biomedical Data Science

Background:

• Electronic health records (EHRs) offer valuable clinical prediction capabilities.
• Current EHR explanations are often post hoc and lack patient-level decision utility.
• Integrating structured knowledge with EHR data is crucial for interpretable AI.

Purpose of the Study:

• To develop a knowledge-graph (KG)-guided chain-of-thought (CoT) framework for visit-level disease prediction using EHRs.
• To enhance the interpretability and accuracy of clinical prediction models.
• To evaluate the framework's performance against traditional methods and its zero-shot transferability.

Main Methods:

• Mapping ICD-9 codes to PrimeKG to mine disease-relevant knowledge graph paths.
• Scaffolding temporally consistent CoT explanations using mined KG paths.
• Fine-tuning lightweight large language models (LLMs) like LLaMA-3.1-Instruct-8B and Gemma-7B on MIMIC-III data.
• Evaluating model performance using AUROC and macro-AUPR metrics and assessing zero-shot performance on the CRADLE cohort.

Main Results:

• The KG-guided CoT framework achieved AUROC of 0.66-0.70 and macro-AUPR of 0.40-0.47 on MIMIC-III data.
• Models demonstrated significant zero-shot transferability to the CRADLE cohort, improving accuracy from 0.40-0.51 to 0.72-0.77.
• Blinded clinicians preferred the KG-guided CoT explanations for clarity, relevance, and correctness over baseline methods.

Conclusions:

• The proposed KG-guided CoT framework effectively improves visit-level disease prediction accuracy from EHRs.
• The framework generates interpretable explanations that are highly valued by clinicians.
• This approach represents a significant advancement in developing trustworthy and actionable AI for clinical decision support.