Large Language Model-Supported Identification of Intellectual Disabilities in Clinical Free-Text Summaries: Mixed...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Genetics

Background:

• Free-text clinical data offer rich patient information but pose challenges for research-quality phenotype extraction.
• Manual extraction is time-consuming and unsuitable for large datasets.
• Automated methods are hindered by a lack of annotated resources.

Purpose of the Study:

• To develop a large language model (LLM) pipeline using few-shot learning for extracting clinical information from free-text summaries.
• To assess the pipeline's performance in classifying comorbid intellectual disability (ID) in patients with severe mental illness.
• To perform genetic validation of LLM-identified ID cases.

Main Methods:

• Developed a two-stage classification approach combining information extraction (IE) and human-in-the-loop techniques.
• Evaluated Fine-Tuned Language Text-To-Text Transfer Transformer (Flan-T5) and Large Language Model Architecture (LLaMA) models.
• Used 1144 clinical summaries (314 annotated) and genetic data from 547 individuals for validation.

Main Results:

• A two-stage approach with manual validation effectively identified suspected ID from free-text records with minimal training data.
• The Flan-T5 model with IE achieved an F1-score of 0.867.
• LLM-identified ID cases showed significant enrichment for de novo variants in developmental disorder risk genes (OR 29.1, P=2.1×10-5).

Conclusions:

• LLMs, in-context learning, and human-in-the-loop methods enhance clinical data extraction and categorization.
• LLMs can identify patients with severe mental illness and suspected ID, a meaningful subgroup.
• This proof-of-concept study demonstrates LLMs' utility in clinical applications.