External Validation of Postpartum Hemorrhage Prediction Models Using Electronic Health Record Data

Area of Science:

• Obstetrics and Gynecology
• Data Science in Healthcare
• Clinical Prediction Models

Background:

• Machine learning models demonstrated high accuracy in predicting postpartum hemorrhage (PPH) using estimated blood loss (EBL) in older datasets.
• These models were developed on data from 2002-2008 and defined PPH using EBL ≥1,000 mL.
• There is a need to validate these models in contemporary cohorts using quantitative blood loss (QBL) methods.

Purpose of the Study:

• To externally validate existing machine learning models for PPH prediction in a more recent cohort.
• To assess the performance of models using quantitative blood loss (QBL) measurements.
• To develop and evaluate new PPH prediction models using QBL data.

Main Methods:

• Utilized electronic health record (EHR) data from 5,261 deliveries between February 2019 and May 2020.
• Mapped 55 predictors from previously published Consortium for Safe Labor (CSL) models to the EHR data.
• Defined PPH as QBL ≥1,000 mL within 24 hours; assessed model discrimination and calibration, and refit models using study cohort data.

Main Results:

• The rate of PPH was significantly higher in the study cohort (25%) compared to the original CSL cohort (4.8%), potentially due to measurement differences.
• Original CSL models showed lower discrimination in the QBL cohort (C-statistic up to 0.57).
• Refit models using QBL data achieved improved discrimination (C-statistic up to 0.64) and calibration.

Conclusions:

• The accuracy of established PPH prediction models decreased when applied to a contemporary cohort using QBL methods.
• Differences in blood loss measurement (EBL vs. QBL) significantly impact model performance.
• Further research is needed to refine PPH prediction models as QBL adoption increases.