A phenotype-based AI pipeline outperforms human experts in differentially diagnosing rare diseases using EHRs

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Genomics and Rare Diseases

Background:

• Rare diseases affect approximately 350 million people globally, presenting diagnostic challenges due to physician inexperience and disease complexity.
• Accurate and timely diagnosis of rare diseases is critical for effective patient management and treatment.
• Current diagnostic approaches often struggle with the vast number of rare conditions and subtle phenotypic variations.

Purpose of the Study:

• To introduce PhenoBrain, a novel, fully automated artificial intelligence (AI) pipeline for rare disease diagnosis.
• To evaluate PhenoBrain's performance in differential diagnosis using extensive, multi-country rare disease datasets.
• To compare PhenoBrain's diagnostic accuracy against existing prediction methods, specialist physicians, and large language models.

Main Methods:

• PhenoBrain employs a BERT-based natural language processing (NLP) model to extract patient phenotypes from electronic health records (EHRs).
• The pipeline integrates five newly developed diagnostic models for rare disease differential diagnosis.
• The AI system was trained and validated on a diverse dataset of 2271 cases across 431 rare diseases from multiple countries.

Main Results:

• PhenoBrain achieved a top-3 recall of 0.513 and a top-10 recall of 0.654 on 1936 test cases, outperforming 13 leading prediction methods.
• In a human-computer study, PhenoBrain demonstrated superior performance (top-3 recall: 0.613, top-10 recall: 0.813) compared to 50 specialist physicians and advanced LLMs.
• Integrating PhenoBrain's predictions with specialist input improved top-3 recall to 0.768, highlighting synergistic potential.

Conclusions:

• PhenoBrain represents a significant advancement in AI-driven rare disease diagnosis, offering automated phenotype extraction and accurate differential diagnoses.
• The AI system surpasses current state-of-the-art prediction tools and human expert performance in identifying rare diseases.
• PhenoBrain has the potential to substantially enhance diagnostic accuracy and efficiency within clinical workflows for rare diseases.