Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notes.

Area of Science:

• Genomics
• Artificial Intelligence
• Clinical Informatics

Background:

• Large language models (LLMs) often struggle with phenotype-driven gene prioritization for rare diseases using standardized terms.
• Real-world clinical applications require LLMs to process unstructured clinical notes, not just Human Phenotype Ontology (HPO) terms.
• A significant challenge is instructing LLMs to predict candidate genes or diagnoses from raw clinical text.

Purpose of the Study:

• To develop and evaluate novel methods for improving gene prioritization in rare diseases using LLMs and clinical notes.
• To assess the efficacy of combining Chain-of-Thought (CoT) and Retrieval Augmented Generation (RAG) for analyzing unstructured clinical data.

Main Methods:

• Introduced two hybrid approaches: RAG-driven CoT and CoT-driven RAG, integrating CoT reasoning with RAG data retrieval.
• Utilized a five-question CoT protocol to mimic expert clinical reasoning.
• Employed RAG to retrieve relevant information from knowledge bases like HPO and Online Mendelian Inheritance in Man (OMIM).
• Evaluated methods on diverse rare disease datasets: 5,980 Phenopacket-derived notes, 255 literature narratives, and 220 clinical notes.

Main Results:

• Recent LLMs (e.g., Llama 3.3-70B-Instruct, DeepSeek-R1-Distill-Llama-70B) showed improved performance over older models (Llama 2, GPT-3.5).
• Both RAG-driven CoT and CoT-driven RAG significantly outperformed baseline foundation models in gene prioritization from clinical notes.
• The DeepSeek backbone achieved over 40% top-10 gene accuracy on Phenopacket-derived notes using these hybrid methods.
• RAG-driven CoT excelled with high-quality notes, while CoT-driven RAG was advantageous for lengthy, noisy clinical text.

Conclusions:

• Hybrid CoT and RAG methods offer a robust solution for candidate gene prioritization using unstructured clinical notes in rare disease diagnosis.
• These approaches enhance the utility of LLMs in clinical settings by bridging the gap between raw text and structured medical knowledge.
• The choice between RAG-driven CoT and CoT-driven RAG depends on the quality and nature of the input clinical notes.