Validation of a prediction model for postpartum hospital use in geographic contexts with greater rural representation

The postpartum period is a critical window to address maternal health inequities. Black, Hispanic, Indigenous, and rural populations experience disproportionately high rates of postpartum morbidity and postpartum hospital use (PHU), defined as readmissions or emergency department (ED) visits after delivery. Delivery hospitalizations provide an opportunity for early identification of individuals at high risk of PHU, who may benefit from targeted interventions to prevent adverse outcomes. We previously developed a 30-day PHU prediction model using New York City (NYC) birth data (2016-2018), which achieved an area under the receiver operating curve (AUC) of 0.69. However, its performance in obstetric populations outside of a dense urban setting has not been examined.

We aimed to evaluate the accuracy of our PHU prediction model in South Carolina (SC) and Florida (FL), states with diverse populations, including substantial rural representation, and in a different US geographic region than the NYC development sample. We additionally examined model performance in subgroups defined by race/ethnicity, Medicaid insurance, and rural residence.

We performed a retrospective cohort study of linked birth certificate and hospital discharge data from 2016 to 2019 births in SC (n=183,836) and FL (n=696,963). We ascertained 21 predictors consistent with the NYC model, excluding two variables (prenatal depression, Apgar) unavailable in the new states. PHU was defined as ≥1 inpatient or ED encounter within 30 days postpartum. Model performance was assessed using calibration (intercept, slope) and discrimination (AUC). We first applied the original NYC model coefficients to generate PHU predicted probabilities among SC and FL births. We then tested a series of stepwise model updating strategies: recalibrating intercepts, re-estimating predictor coefficients, and incorporating additional contextual indicators of hospital access-residential rurality and driving distance to the delivery hospital-hypothesized to be relevant in settings with larger rural populations.

Cumulative 30-day PHU incidence was 7.4% in SC and 7.2% in FL; rates were higher among Black individuals, Medicaid-insured individuals, and rural residents. Applying the original NYC model coefficients achieved an AUC of 0.68 (95% CI 0.67-0.68) and 0.69 (95% CI 0.68-0.69) among SC and FL births, respectively, but generated overestimated and extreme risk predictions compared with observed risk. Updating model intercepts corrected calibration, and additionally re-estimating coefficients resulted in an AUC of 0.69 (95% CI 0.68-0.69) in SC and 0.70 (95% CI 0.70-0.71) in FL. Inclusion of hospital distance and rurality did not meaningfully change calibration or discrimination. Model discrimination was slightly lower when subset to Black, Medicaid-insured, and rural residents, but AUC increased within each group after re-estimating predictor coefficients.

A PHU prediction model developed in an urban NYC cohort demonstrated similarly moderate discrimination in SC and FL as in the original NYC sample, but overestimated absolute risk in these new settings. Modest model updating, including recalibration of intercepts and re-estimation of coefficients, yielded well-calibrated models without requiring new predictors. Hospital access measures did not substantially improve prediction. These findings demonstrate that an existing prediction model for postpartum acute care use can be adapted for use in geographically and socio-demographically diverse populations. Geographic validation and model updating are important steps in deploying predictive tools to reduce persistent gaps in maternal health outcomes.