Fine-tuning large language models for rare disease concept normalization

Area of Science:

• Computational biology
• Medical informatics
• Natural Language Processing

Background:

• Rare disease concept normalization is crucial for clinical data analysis.
• Existing methods struggle with the complexity and variability of phenotype terms.
• Large language models (LLMs) offer potential but require domain-specific adaptation.

Purpose of the Study:

• To develop a novel method for rare disease concept normalization.
• To fine-tune the Llama 2 LLM using a corpus from the Human Phenotype Ontology (HPO).
• To evaluate the performance of fine-tuned models in normalizing phenotype terms.

Main Methods:

• Generated two corpora: HPO names with identifiers (NAME) and names with synonyms and identifiers (NAME+SYN).
• Fine-tuned Llama 2 (Llama2-7B) on these corpora.
• Evaluated models using various phenotype terms, including those with typos and unseen synonyms.

Main Results:

• Fine-tuned models achieved over 99% accuracy when terms were in the fine-tuning corpora.
• NAME+SYN model accuracy reached 92.7% for unseen HPO synonyms, significantly outperforming NAME (11.2%).
• Performance improved with typo-specific fine-tuning, reaching 61.8% accuracy for NAME+SYN.

Conclusions:

• Fine-tuned Llama 2 models can normalize diverse phenotype terms, including misspellings and synonyms.
• This approach enables effective use of LLMs for identifying and normalizing medical entities in clinical narratives.
• The method provides a robust solution for mapping clinical terms to controlled vocabularies.