Development and external validation of an ICU mortality prediction model using routinely collected clinical data

Accurate early prediction of intensive care unit (ICU) mortality is important for risk stratification and resource allocation. However, many prediction models show reduced performance in external populations. We conducted a multicenter retrospective cohort study to develop and externally validate an ICU mortality model using routinely collected variables from the first 24 hours of ICU admission. A prespecified multivariable logistic regression model was developed in the Medical Information Mart for Intensive Care IV and externally validated in the eICU Collaborative Research Database. As an exploratory comparison, we also evaluated an extreme gradient boosting model using the same predictor set. Performance was assessed by discrimination, calibration, and decision-curve analysis. The derivation cohort included 39,627 patients (Medical Information Mart for Intensive Care IV), and the external cohort included 27,821 patients (eICU Collaborative Research Database). The logistic model showed good derivation performance (area under the receiver operating characteristic curve [AUROC] 0.778, area under the precision-recall curve [AUPRC] 0.353, and Brier 0.075) and adequate external discrimination (AUROC 0.762, AUPRC 0.375, and Brier 0.096), but poor external calibration (slope 0.17). In exploratory comparison, extreme gradient boosting showed improved external performance (AUROC 0.814, 95% confidence interval 0.807-0.821; AUPRC 0.438; and Brier 0.089) and better calibration metrics (slope 0.84; expected-to-observed ratio 0.93). The primary interpretable logistic model supports early ICU risk stratification but requires local recalibration before transport to new settings. A comparative nonlinear model showed better external predictive performance, but prospective validation and clinical impact evaluation are required before routine implementation.