Knowledge graph construction for heart failure using large language models with prompt engineering.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Knowledge Representation

Background:

• Accurate disease knowledge graphs are vital for clinical applications, but traditional BERT-based methods require extensive data, which is often scarce in real-world medical settings.
• Existing models struggle with recognizing novel entities and relationships not present in training data, limiting their practical utility.

Purpose of the Study:

• To develop a practical pipeline for constructing a heart failure knowledge graph.
• To leverage large language models and expert refinement for enhanced knowledge graph construction.
• To address the limitations of data scarcity and out-of-distribution entity recognition in medical knowledge graph creation.

Main Methods:

• A novel pipeline integrating large language models (LLMs) with medical expert refinement was developed.
• Prompt engineering was applied to schema design, information extraction, and knowledge completion phases.
• The TwoStepChat approach, combined with task-specific prompt templates, was utilized for optimal performance.

Main Results:

• The TwoStepChat method demonstrated superior performance compared to vanilla prompting and fine-tuned BERT-based baselines.
• The proposed pipeline significantly reduces annotation time, saving 65% compared to manual annotation.
• The method effectively extracts out-of-distribution information, addressing a key limitation of existing approaches.

Conclusions:

• The developed pipeline offers an efficient and effective solution for constructing specialized medical knowledge graphs.
• LLMs and expert refinement, particularly with the TwoStepChat approach, show promise for overcoming data limitations in medical AI.
• This work contributes to advancing clinical decision support and health management through improved knowledge representation.