SynthEHR-eviction: enhancing eviction SDoH detection with LLM-augmented synthetic EHR data

Area of Science:

• Health Informatics
• Natural Language Processing
• Social Determinants of Health

Background:

• Eviction is a critical social determinant of health (SDoH) linked to housing instability, unemployment, and mental health issues.
• Information on eviction is often unstructured in electronic health records (EHRs), hindering its use in research and clinical practice.
• Existing methods for extracting SDoH from clinical notes are limited, especially for low-resource scenarios.

Purpose of the Study:

• To introduce SynthEHR-Eviction, a scalable pipeline for extracting eviction-related SDoH from clinical notes.
• To create a large, publicly available dataset of eviction-related SDoH with 14 fine-grained categories.
• To evaluate the performance of fine-tuned large language models (LLMs) on this dataset and compare them to existing models.

Main Methods:

• Developed SynthEHR-Eviction, a pipeline integrating human-in-the-loop annotation, automated prompt optimization (APO), and reasoning-augmented fine-tuning.
• Created a public dataset of eviction-related SDoH from clinical notes.
• Fine-tuned LLMs (Qwen2.5, LLaMA3) on the SynthEHR-Eviction dataset and evaluated performance using Macro-F1 scores.
• Compared fine-tuned LLMs against GPT-4o-APO, GPT-4o-mini-APO, and BioBERT.

Main Results:

• Fine-tuned LLMs achieved high performance: 88.8% Macro-F1 for eviction and 90.3% for other SDoH.
• SynthEHR-Eviction outperformed GPT-4o-APO, GPT-4o-mini-APO, and BioBERT in SDoH extraction.
• The pipeline reduced annotation effort by over 80% and accelerated dataset creation.
• Achieved cost-effective deployment across various model sizes.

Conclusions:

• SynthEHR-Eviction enables scalable and accurate detection of eviction-related SDoH from clinical notes.
• The developed dataset and fine-tuned LLMs advance SDoH research and application in healthcare.
• The pipeline demonstrates generalizability to other information extraction tasks and reduces resource requirements.