Development and external validation of an ICU mortality prediction model using routinely collected clinical data

Area of Science:

• Critical Care Medicine
• Health Informatics
• Biostatistics

Background:

• Accurate prediction of intensive care unit (ICU) mortality is vital for patient management and resource allocation.
• Existing ICU mortality prediction models often exhibit decreased performance when applied to new patient populations.
• Routinely collected data within the first 24 hours of ICU admission offers a potential window for early risk stratification.

Purpose of the Study:

• To develop and externally validate an ICU mortality prediction model using data from the initial 24 hours of ICU admission.
• To compare the performance of a multivariable logistic regression model with an extreme gradient boosting model for ICU mortality prediction.
• To assess the generalizability and calibration of developed models in an external patient cohort.

Main Methods:

• A multicenter retrospective cohort study design was employed.
• A logistic regression model was developed using the Medical Information Mart for Intensive Care IV (MIMIC-IV) database and validated on the eICU Collaborative Research Database.
• An extreme gradient boosting model was explored as a comparative nonlinear approach using the same predictor set.

Main Results:

• The logistic regression model demonstrated strong performance in the derivation cohort (AUROC 0.778) and adequate discrimination in the external cohort (AUROC 0.762) but poor external calibration (slope 0.17).
• The exploratory extreme gradient boosting model exhibited superior external performance (AUROC 0.814) and improved calibration metrics (slope 0.84).
• The logistic model requires local recalibration for new settings, while the nonlinear model warrants further validation.

Conclusions:

• The developed interpretable logistic model can aid in early ICU risk stratification but necessitates local recalibration for external use.
• A comparative nonlinear model demonstrated enhanced external predictive performance, suggesting potential benefits for ICU mortality prediction.
• Prospective validation and clinical impact assessment are essential before the routine implementation of advanced predictive models in critical care settings.