Improving automated deep phenotyping through large language models using retrieval-augmented generation.

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Accurate diagnosis of rare genetic disorders requires precise phenotypic and genotypic analysis.
• Human Phenotype Ontology (HPO) provides a standardized language for clinical phenotypes.
• Existing rule-based HPO extraction tools and large language models (LLMs) have limitations in accuracy and reliability.

Purpose of the Study:

• To introduce RAG-HPO, a Python-based tool leveraging retrieval-augmented generation (RAG) to enhance LLM-based HPO term assignment accuracy.
• To overcome limitations of baseline LLMs and eliminate the need for extensive fine-tuning.

Main Methods:

• RAG-HPO integrates a dynamic vector database of over 54,000 phenotypic phrases mapped to HPO IDs.
• Clinical text is processed by an LLM to extract phenotypic phrases.
• Extracted phrases are semantically matched against the vector database for real-time retrieval and contextual matching.
• Best term matches are returned to the LLM for final HPO term assignment.

Main Results:

• RAG-HPO + LLaMa-3.1 70B achieved a mean precision of 0.81, recall of 0.76, and F1 score of 0.78, significantly outperforming conventional tools (p < 0.00001).
• Out of 1648 returned terms, 19.1% were false positives, with less than 1% being hallucinations.
• The majority of false positives were broader ancestor terms, potentially useful in certain contexts.

Conclusions:

• RAG-HPO is a user-friendly and adaptable tool that enhances phenotypic analysis for rare diseases.
• The tool significantly improves precision and recall in HPO term assignment, accelerating genetic research and clinical genomics.
• RAG-HPO is publicly available at https://github.com/PoseyPod/RAG-HPO.