Using GPT-4 to annotate the severity of all phenotypic abnormalities within the human phenotype ontology

The Human Phenotype Ontology (HPO) provides a unified framework cataloguing over 17,500 phenotypic abnormalities across more than 8,600 rare diseases, defining hierarchical relationships between them. For example, classifying missing arms and missing legs as both abnormalities of the limb. This structure enables phenome-wide analyses, including the prioritisation of phenotypes as candidates for gene therapy. However, the HPO currently lacks sufficient metadata describing the clinical severity of these phenotypes. Manual expert curation at this scale would be prohibitively labour-intensive, creating a need for automated approaches to systematically annotate phenotypic severity.

GPT-4, a large language model (LLM) developed by OpenAI, was employed to annotate the severity of all phenotypic abnormalities catalogued in the HPO. Severity was operationalised using nine clinical characteristics: congenital onset, reduced fertility, sensory impairments, impaired mobility, immunodeficiency, physical malformations, cancer, intellectual disability, and death. Each characteristic was further qualified by frequency of occurrence across four levels: never, rarely, often, and always. To assess annotation quality, GPT-4's outputs were benchmarked against ground-truth labels embedded within the HPO itself. For instance, phenotypes residing in the "Cancer" HPO branch were expected to be annotated as cancer-causing. A novel severity scoring system was then developed that integrates both the nature of each clinical characteristic and its frequency of occurrence.

Benchmarking demonstrated strong performance across all clinical characteristics, with true positive recall rates ranging from 89% to 100% (mean = 97%). This indicates that GPT-4 can replicate expert-level curation with high fidelity. The resulting severity scoring system produced quantitative severity metrics for phenotypic abnormalities across the HPO, incorporating both the type and frequency of associated clinical characteristics.

These findings demonstrate that LLMs can automate the large-scale curation of clinical metadata with a high degree of accuracy, substantially reducing the burden of manual expert annotation. The severity metrics generated here provide a foundation for systematically ranking human phenotypes by their impact on health and quality of life, enabling more principled prioritisation of targets for therapeutic intervention, particularly in the context of rare diseases where evidence is sparse and resources for curation are limited. Future work may extend this framework to incorporate additional clinical dimensions or validate annotations against independent clinical datasets.