ChatGPT for phenotypes extraction: one model to rule them all?

Area of Science:

• Natural Language Processing (NLP)
• Genomic Medicine
• Machine Learning

Background:

• Information Extraction (IE) identifies factual knowledge from unstructured text for downstream applications.
• Accurate phenotype extraction from clinical reports is vital for improving genetic disease diagnosis via deep phenotyping.
• Large Language Models (LLMs) are increasingly used for phenotype extraction, with recent advancements in sophisticated techniques.

Purpose of the Study:

• To rigorously evaluate ChatGPT's performance in phenotype extraction compared to state-of-the-art solutions.
• To discuss the potential impacts and technical evolutions of LLMs in the medical domain for phenotype extraction.
• To assess the relevance of generic instruction-oriented LLMs against specialized NLP approaches for clinical text.

Main Methods:

• Comparative evaluation of ChatGPT and existing state-of-the-art Information Extraction (IE) pipelines.
• Focus on phenotype concept recognition from free-text clinical notes.
• Analysis of performance metrics for accuracy in extracting phenotypic information.

Main Results:

• ChatGPT's performance in phenotype extraction is rigorously evaluated against current state-of-the-art methods.
• The study provides insights into the effectiveness of generic LLMs versus specialized NLP techniques for clinical data.
• Identifies potential impacts and necessary technical evolutions for LLM applications in medical IE.

Conclusions:

• The study offers a critical comparison of ChatGPT and existing solutions for clinical phenotype extraction.
• Findings are essential for advancing deep phenotyping and improving genetic diagnosis.
• Highlights the need for continued research into LLM applications within genomic medicine.