Predicting Behavioral Determinants of Health from Clinical Text Using Transformer Models and BiLSTM

Area of Science:

• Health Informatics
• Natural Language Processing
• Clinical Text Analysis

Background:

• Social and behavioral determinants of health (BDoH) significantly impact patient outcomes.
• BDoH information is often unstructured in clinical text, limiting its utility.
• Accurate detection of BDoH is crucial for improved patient understanding and decision-making.

Purpose of the Study:

• To enhance the prediction of BDoH from medical records.
• To systematically compare transformer-based models (Bio-ClinicalBERT, BioBERT, BioMedBERT, RoBERTa, T5-EHR) with and without BiLSTM.
• To address class imbalance and evaluate generative vs. discriminative models for BDoH classification.

Main Methods:

• Evaluated five transformer models combined with BiLSTM on the MIMIC-III dataset.
• Compared oversampling, undersampling, and class weighting for class imbalance.
• Assessed standalone model performance and BiLSTM integration for sequential modeling.
• Benchmarked against published results using precision, recall, and F1 scores.

Main Results:

• Class weighting proved most effective for handling class imbalance across all models.
• BiLSTM integration improved Bio-ClinicalBERT, BioMedBERT, and BioBERT performance.
• RoBERTa and T5-EHR performed strongly as standalone models, without BiLSTM benefit.
• T5-EHR achieved the highest F1 scores across all labels, surpassing prior baselines.

Conclusions:

• Effective handling of class imbalance is essential for robust BDoH prediction.
• BiLSTM benefits specific models by capturing sequential dependencies.
• RoBERTa and T5-EHR demonstrate the strength of their pretrained representations.
• The T5-EHR model, adapted for clinical text, sets a new state-of-the-art for BDoH classification.