An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontology-Enhanced Large...

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Rare Disease Research

Background:

• Rare diseases affect millions globally but lack specific diagnostic codes (ICD-9, ICD-10), hindering patient identification and research.
• Current large language models (LLMs) lack specialized medical knowledge for effective rare disease data management and extraction.
• Challenges in rare disease identification complicate clinical trial recruitment and research efforts.

Purpose of the Study:

• To develop an automated system (AutoRD) for extracting rare disease information from medical text.
• To integrate ontologies and structured knowledge for superior rare disease entity and relation extraction.
• To surpass the performance of common LLMs and traditional methods in rare disease mining.

Main Methods:

• Developed AutoRD as a pipeline: data preprocessing, entity/relation extraction, entity calibration, and knowledge graph construction.
• Utilized GPT-4 and medical knowledge graphs from Human Phenotype and Orphanet ontologies.
• Employed chain-of-thought reasoning and prompt engineering for enhanced extraction.

Main Results:

• AutoRD achieved an overall entity extraction F1-score of 56.1% and relation extraction F1-score of 38.6% on the RareDis2023 dataset.
• Demonstrated a 14.4% improvement over baseline LLMs in overall extraction performance.
• Achieved a high F1-score of 83.5% for rare disease entity extraction, showcasing precision and recall.

Conclusions:

• AutoRD effectively extracts rare disease information and builds knowledge graphs, addressing LLM limitations in this domain.
• Ontology-enhanced LLMs significantly improve rare disease identification and connection to clinical features.
• The system facilitates better patient identification for research and clinical trial recruitment, promoting inclusivity.