SynthEHR-Eviction: Enhancing Eviction SDoH Detection with LLM-Augmented Synthetic EHR Data

Area of Science:

• Health Informatics
• Natural Language Processing
• Social Determinants of Health

Background:

• Eviction is an understudied social determinant of health (SDoH) associated with adverse health outcomes.
• Eviction data is often unstructured in electronic health records (EHRs), hindering its use in research and clinical practice.

Purpose of the Study:

• To introduce SynthEHR-Eviction, a scalable pipeline for extracting eviction statuses from clinical notes.
• To create the largest public dataset of eviction-related SDoH using the developed pipeline.
• To evaluate the performance of fine-tuned large language models (LLMs) on this dataset.

Main Methods:

• Developed SynthEHR-Eviction, a pipeline integrating LLMs, human-in-the-loop annotation, and automated prompt optimization (APO).
• Created a public dataset of eviction-related SDoH with 14 fine-grained categories.
• Trained and evaluated fine-tuned LLMs (Qwen2.5, LLaMA3) and compared them against other models (GPT-4o-APO, GPT-4o-mini-APO, BioBERT).

Main Results:

• Fine-tuned LLMs trained on SynthEHR-Eviction achieved high Macro-F1 scores: 88.8% for eviction and 90.3% for other SDoH.
• The pipeline demonstrated superior performance compared to GPT-4o-APO, GPT-4o-mini-APO, and BioBERT.
• Annotation effort was reduced by over 80%, accelerating dataset creation.

Conclusions:

• SynthEHR-Eviction enables scalable and cost-effective detection of eviction-related SDoH from clinical notes.
• The developed dataset and fine-tuned models advance research on the health impacts of eviction.
• The pipeline's methodology is generalizable to other information extraction tasks in healthcare.