RDguru: A Conversational Intelligent Agent for Rare Diseases.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Rare Disease Diagnostics

Background:

• Large language models (LLMs) show promise in healthcare but struggle with accuracy, especially in rare diseases (RDs).
• Existing diagnostic tools for RDs lack comprehensive integration and advanced AI capabilities.
• Clinical adoption of LLMs is limited by concerns over erroneous and harmful outputs.

Purpose of the Study:

• To introduce RDguru, a conversational AI agent designed to enhance rare disease diagnosis and knowledge Q&A.
• To develop a novel multi-source fusion diagnostic model for improved diagnostic recall.
• To evaluate RDguru's performance against established benchmarks and real-world patient data.

Main Methods:

• RDguru utilizes the LangChain framework and GPT-3.5-turbo, integrating authoritative knowledge sources.
• A deep Q-network-based diagnostic model fuses GPT-4, PheLR, and phenotype matching strategies.
• Retrieval-augmented generation (RAG) techniques are employed for knowledge Q&A, phenotype annotation, and differential diagnosis (DDX).

Main Results:

• RDguru demonstrated superior performance over ChatGPT in knowledge Q&A, validated by ROUGE, GPT-4, and RAGAs metrics.
• The multi-source fusion model achieved a 63.87% recapture rate for actual diagnoses in 238 RD cases, a 5.47% improvement over PheLR.
• RDguru's consultation strategy effectively elicited diagnostically relevant phenotypes and improved diagnosis prioritization.

Conclusions:

• RDguru offers a reliable and effective solution for rare disease knowledge Q&A and differential diagnosis.
• The developed multi-source fusion model significantly enhances diagnostic accuracy in rare diseases.
• RDguru shows strong potential to improve clinical decision-making and support for rare disease patients.