Automating clinical phenotyping using natural language processing

Area of Science:

• Medical Informatics
• Computational Biology
• Health Data Science

Background:

• Electronic health record (EHR) analysis for patient phenotyping is often manual and labor-intensive.
• Scalable and automated methods are needed to efficiently extract clinical information from EHRs.

Purpose of the Study:

• To develop and compare rule-based (spaCy) and Large Language Model (LLM)-based (GPT-4) computable phenotyping algorithms.
• To sub-phenotype Crohn's disease patients based on age at diagnosis and disease behavior using clinical narrative text.

Main Methods:

• Utilized spaCy for rule-based phenotyping and GPT-4 for LLM-based phenotyping.
• Analyzed 49,572 clinical notes and 2204 radiology reports from 584 Crohn's disease patients.
• Evaluated algorithm performance using F1 scores, recall, precision, and specificity on sentence and patient levels.

Main Results:

• GPT-4 demonstrated comparable or superior performance to rule-based methods.
• Note-level F1 scores reached at least 0.90 for disease behavior and 0.82 for age at diagnosis.
• Patient-level F1 scores were at least 0.66 for disease behavior and 0.71 for age at diagnosis.

Conclusions:

• This is the first study comparing LLMs and rule-based systems for complex Crohn's disease sub-phenotyping from clinical text.
• LLM performance showed no statistical difference compared to human experts.
• LLMs hold significant potential for large-scale EHR analysis and streamlining chart review processes.