Fine-tuning Large Language Models for Rare Disease Concept Normalization.

Area of Science:

• Natural Language Processing
• Bioinformatics
• Medical Informatics

Background:

• Rare disease concept normalization is crucial for clinical data analysis.
• Existing methods struggle with variations in phenotype term representation.

Purpose of the Study:

• To develop a novel method for rare disease concept normalization using fine-tuned Llama 2.
• To improve the accuracy of identifying Human Phenotype Ontology (HPO) identifiers from clinical narratives.

Main Methods:

• Fine-tuning Llama 2 (Llama2-7B) with two domain-specific corpora derived from HPO: HPO names (NAME) and HPO names with synonyms (NAME+SYN).
• Evaluating model performance using various phenotype terms, including those with typos and unseen synonyms.
• Comparing fine-tuned models against ChatGPT-3.5.

Main Results:

• Fine-tuned models achieved over 99% accuracy when phenotype terms were present in the fine-tuning corpora.
• The NAME+SYN model demonstrated 92.7% accuracy for unseen HPO synonyms, significantly outperforming ChatGPT-3.5 (~20%).
• The NAME+SYN model showed improved accuracy (61.8%) with typo-specific fine-tuning.

Conclusions:

• Fine-tuned Llama 2 models can effectively normalize phenotype terms, including misspellings and synonyms not present in the training data.
• This approach offers a robust solution for using large language models to extract and normalize medical entities from clinical text.
• The study highlights the potential of domain-specific fine-tuning for enhancing LLM performance in biomedical applications.