Related Experiment Video
Updated: May 21, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Extracting Social Determinants of Health From Electronic Health Records: Development and Comparison of Rule-Based and
Bo Wang1,2,3,4, Dia Kabir1,2,3, Cheryl Renee Clark5
1Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA, United States.
JMIR Medical Informatics
|May 19, 2026
Summary
Extracting social determinants of health (SDoH) from clinical notes is crucial. Large language models (LLMs) like GPT-5 and GPT-4o, especially when combined with rule-based systems, show superior performance in identifying SDoH compared to traditional methods.
Area of Science:
- Natural Language Processing (NLP)
- Clinical Informatics
- Population Health Research
Background:
- Social determinants of health (SDoH) significantly impact health outcomes.
- SDoH data are often underrepresented in structured electronic health records (EHRs).
- Unstructured clinical notes are a rich source of SDoH information.
Purpose of the Study:
- To develop and evaluate cost-efficient methods for extracting SDoH from clinical text.
- To compare rule-based NLP and large language model (LLM) approaches for SDoH extraction.
- To assess the performance of various GPT models and ensemble methods.
Main Methods:
- Construction of a gold-standard annotated corpus of clinical text from 171 patients.
- Development and evaluation of a rule-based system (RBS).
- Assessment of 7 OpenAI GPT models (GPT-4o, GPT-4.1, GPT-4.1-mini, o4-mini, GPT-5, GPT-5-mini, GPT-3) in zero-shot and few-shot settings.
- Implementation of late-fusion ensemble methods combining RBS and LLM outputs.
Main Results:
- RBS achieved high precision (0.96) but low recall (0.68) for SDoH domains.
- GPT models consistently outperformed RBS in recall and F1-scores.
- GPT-5 and GPT-5-mini achieved the best domain-level performance (F1=0.89) in few-shot settings.
- o4-mini showed the highest subcategory-level performance (F1=0.88).
- Late-fusion ensemble improved domain-level performance (F1=0.92) with balanced precision (0.93) and recall (0.90).
Conclusions:
- Advanced GPT models, including mini versions, demonstrate strong SDoH extraction performance without fine-tuning.
- LLM-based methods consistently outperform rule-based NLP systems for SDoH extraction.
- Integrating rule-based and LLM methods via late fusion enhances domain-level extraction.
- A cost-efficient framework for accurate SDoH identification from clinical text is established.
- This facilitates advancements in population health research and clinical informatics.
Related Concept Videos
01:25Statistical Methods for Analyzing Epidemiological Data
Epidemiological data primarily involves information on specific populations' occurrence, distribution, and determinants of health and diseases. This data is crucial for understanding disease patterns and impacts, aiding public health decision-making and disease prevention strategies. The analysis of epidemiological data employs various statistical methods to interpret health-related data effectively. Here are some commonly used methods:
01:30Methods of Documentation VII: EMR
Electronic Medical Records (EMRs) primarily center around electronically documenting patients' health information within a single healthcare organization or practice. They contain essential clinical data related to a patient's medical history, diagnoses, medications, treatment plans, lab results, and other pertinent information relevant to the specific encounter or episode of care. EMRs are designed to streamline documentation and workflow processes within individual healthcare settings,...
01:25Models of Health Promotion and Illness Prevention I
A model is a theoretical way to understand a concept or an idea. Models can overcome barriers to health regardless of diverse economic and cultural backgrounds. In addition, models make the task easier by providing different ways to approach complex issues. There are two major health promotion models: the health belief model and the health promotion model.
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
01:18Steps in Outbreak Investigation
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least squares (OLS)...
01:26Introduction to Epidemiology
Epidemiology, known as the cornerstone of public health, involves studying the distribution and determinants of health-related events in defined populations and applying these insights to control health issues. This is essential for understanding how diseases spread, identifying populations at greater risk, and implementing measures to control or prevent outbreaks. Epidemiology addresses not only infectious diseases but also non-communicable conditions like cancer and cardiovascular disease,...