External Validation of Persistent Severe Acute Kidney Injury Prediction With Machine Learning Model.

Area of Science:

• Nephrology
• Artificial Intelligence in Medicine
• Clinical Informatics

Background:

• Persistent severe acute kidney injury (psAKI) poses a significant clinical challenge.
• Validated prediction models for psAKI are scarce, limiting clinical decision-making.
• Electronic health record data offers potential for real-time risk prediction.

Purpose of the Study:

• To externally validate the Persistent Electronic Alert (PersEA) machine learning model for predicting psAKI.
• To assess the scalability and performance of the PersEA model on an independent dataset.
• To address the scarcity of validated psAKI prediction models.

Main Methods:

• Retrospective analysis of adult intensive care unit patients with stage 2 acute kidney injury (AKI).
• The PersEA model, a boosted tree algorithm, utilized hourly electronic health record data.
• Model performance evaluated using area under the receiver operating characteristic (AUROC) and precision-recall curves (AUPRC), with metrics penalizing late alarms.

Main Results:

• The external validation included 4479 patients from the Mayo Clinic cohort, with 5.22% experiencing psAKI.
• The PersEA model achieved an AUROC of 0.98 (95% CI, 0.97-0.98) and AUPRC of 0.67 (95% CI, 0.60-0.73).
• At a threshold of 0.80 sensitivity, the model demonstrated 0.88 sensitivity, 0.94 specificity, and 0.47 precision on the Mayo Clinic data.

Conclusions:

• The PersEA model demonstrated excellent performance in an external validation cohort.
• The model is scalable and requires minimal tuning on high-quality data.
• This study supports the utility of the PersEA model for real-time psAKI prediction.