Extracting Signs and Symptoms of Hypertensive Disorders in Pregnancy from Clinical Notes Using Natural Language...

Hypertensive disorders in pregnancy (HDP) affect 16% of births in the United States. In this pilot study, we conducted a preliminary evaluation of natural language processing (NLP) in extracting signs and symptoms (SS) of HDP from clinical notes within electronic health records (EHRs).

This retrospective observational pilot study used EHR data from patients admitted for labor and birth (N = 83,003 clinical notes from 17,775 patients). Four SS categories were extracted: elevated blood pressure, neurological, renal, and hepatic/hematologic. Five machine learning models and ClinicalBERT were trained and tested using five-fold cross-validation. The best-performing model was applied to the full dataset. Bivariate analyses were performed to examine (1) differences in HDP diagnoses based on ICD-10 codes (gestational hypertension, preeclampsia, and eclampsia) by SS documentation and (2) differences in SS documentation by patient race and ethnicity.

XGBoost demonstrated the highest macro-average F1-score (0.75). Elevated blood pressure showed the highest F1-score (0.87), followed by neurological SS (0.77). In the full dataset, 24.3% of clinical notes and 42.3% of patients had documentation of at least one SS category. A higher proportion of HDP diagnoses was observed with an increased number of SS categories documented (p < .001). A higher proportion of non-Hispanic Black patients had documentation of SS across all categories.

NLP can extract SS with moderate accuracy, supporting feasibility for larger-scale extraction. Findings also highlight differences in SS documentation by patient race and ethnicity. Future research is needed to improve NLP performance, including expanding annotated data.