Related Experiment Video
Updated: Aug 2, 2025
A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
Natural language processing-driven state machines to extract social factors from unstructured clinical documentation
Katie S Allen1,2, Dan R Hood1, Jonathan Cummins1
1Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA.
This study developed natural language processing (NLP) algorithms to identify social determinants of health, such as housing, financial, and unemployment needs, in clinical notes. The NLP models demonstrated high accuracy and generalizability across different healthcare systems.
Area of Science:
- Medical Informatics
- Computational Linguistics
- Public Health
Background:
- Social determinants of health significantly impact patient outcomes.
- Accurate identification of social needs in clinical settings is crucial for effective intervention.
- Existing methods for extracting social factors from clinical text can be labor-intensive and may lack generalizability.
Purpose of the Study:
- To develop and validate natural language processing (NLP) algorithms for extracting social factors (housing, financial, unemployment) from clinical text.
- To ensure the generalizability of these NLP models across different healthcare systems.
- To establish a scalable method for measuring social needs within institutional settings.
Main Methods:
- Utilized clinical notes from two healthcare systems for model training and validation.
- Employed n-grams and NLP state machines to identify keywords and patterns related to social needs.
- Performed manual review for performance assessment and optimized models through iterative training and evaluation cycles.
- Calculated performance metrics including positive predictive value (PPV), negative predictive value, sensitivity, and specificity.
Main Results:
- Achieved high positive predictive values (PPVs) for housing (0.95), financial (0.89), and unemployment (0.88) after iterative training.
- Validated models on a separate health system, yielding PPVs of 0.94 (housing), 0.97 (financial), and 0.95 (unemployment).
- Demonstrated strong generalizability with specificity scores exceeding 0.95 across all three social factors.
Conclusions:
- Rule-based NLP algorithms effectively identify key social factors within clinical text.
- The developed algorithms exhibit high generalizability and consistent performance across diverse healthcare data.
- These NLP methods offer a valuable tool for systematically measuring social determinants of health in clinical practice.
More Related Videos
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Related Concept Videos
01:15Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...
01:26Methods of Documentation II: POMR
01:29Documentation in Long-Term and Home Healthcare Setting
Long-Term Care Facilities
01:25Formulating and Validating Nursing Diagnosis II
Risk nursing diagnoses represent clinical judgments of an individual, family, or community more vulnerable to developing the health problem than others...
01:18Methods of Documentation I: Source-Oriented Records
In an SOR, each discipline involved in patient care maintains a separate medical record section. This record-keeping method enables easy tracking of patient progress and ensures healthcare staff have access to up-to-date information.
Key Attributes include the following:
01:26Formulating and Validating Nursing Diagnosis I
There are thirteen domains...