Analyzing and Mitigating Model Drift in Acute Kidney Injury Prediction for Hospitalized Patients

Artificial intelligence and machine learning are transforming healthcare by improving clinical risk predictions and diagnostic precision. However, their performance can be compromised by data drifts due to changes in patient populations and evolving clinical practices. This study investigated performance drift in models predicting Acute Kidney Injury (AKI) using electronic health records from 249,749 inpatient encounters over ten years, analyzing performance across both the overall population and nine subgroups with unique health profiles. To mitigate the performance drift, we implemented two model updating strategies: an Overall Population Update (OPU) and a Specific Subgroup Update (SSU). Our results demonstrated significant reductions in drift, with OPU increasing the average area-under-the-precision-recall-curve (AUPRC) by 0.14 in the overall population and 0.11 across subgroups, and SSU improving the average AUPRC by 0.10 among subgroups. These findings highlight the importance of continuous model surveillance and adaptive updates to maintain reliable predictive performance in dynamic clinical environments.