Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Models

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Computational Health

Background:

• Large language models (LLMs) show promise in medical question answering and programming.
• The utility of LLMs for generating interpretable computable phenotypes (CPs) remains largely unexplored.
• Accurate CPs are crucial for scalable clinical decision support, particularly for conditions like hypertension.

Purpose of the Study:

• To evaluate the capability of LLMs in generating accurate and concise CPs for six clinical phenotypes.
• To assess the effectiveness of a novel 'synthesize, execute, debug, instruct' strategy for iterative CP refinement using LLMs.
• To compare LLM-generated CP performance against state-of-the-art machine learning methods.

Main Methods:

• LLMs were employed to generate CPs for six clinical phenotypes of varying complexity.
• A 'synthesize, execute, debug, instruct' strategy was developed and tested, utilizing LLM-generated CPs with data-driven feedback for iterative refinement.
• Zero-shot performance of LLMs was evaluated, alongside their performance after iterative learning.

Main Results:

• LLMs, particularly when enhanced by the iterative learning strategy, demonstrated the ability to generate interpretable and reasonably accurate CPs.
• The performance of LLM-generated CPs approached that of state-of-the-art machine learning methods.
• This approach required significantly fewer training examples compared to traditional methods.

Conclusions:

• LLMs, augmented with iterative refinement strategies, offer a viable approach for generating computable phenotypes.
• This method holds potential for enhancing clinical decision support systems and improving patient care, especially for hypertension.
• LLM-based CP generation can achieve high performance with reduced data requirements, paving the way for scalable solutions.