Machine Learning Models Using Routinely Collected Clinical Data Offer Robust and Interpretable Predictions of 90-Day...

Clinical management of patients receiving immune checkpoint inhibitors (ICIs) could be informed using accurate predictive tools to identify patients at risk of short-term acute care utilization (ACU). We used routinely collected data to develop and assess machine learning (ML) algorithms to predict unplanned ACU within 90 days of ICI treatment initiation.

We used aggregated electronic health record data from 7,960 patients receiving ICI treatments to train and assess eight ML algorithms. We developed the models using pre-SARS-COV-19 COVID-19 data generated between January 2016 and February 2020. We validated our algorithms using data collected between March 2020 and June 2022 (peri-COVID-19 sample). We assessed performance using area under the receiver operating characteristic curves (AUROC), sensitivity, specificity, and calibration plots. We derived intuitive explanations of predictions using variable importance and Shapley additive explanation analyses. We assessed the marginal performance of ML models compared with that of univariate and multivariate logistic regression (LR) models.

Most algorithms significantly outperformed the univariate and multivariate LR models. The extreme gradient boosting trees (XGBT) algorithm demonstrated the best overall performance (AUROC, 0.70; sensitivity, 0.53; specificity, 0.74) on the peri-COVID-19 sample. The algorithm performance was stable across both pre- and peri-COVID-19 samples, as well as ICI regimen and cancer groups. Type of ICI agents, oxygen saturation, diastolic blood pressure, albumin level, platelet count, immature granulocytes, absolute monocyte, chloride level, red cell distribution width, and alcohol intake were the top 10 key predictors used by the XGBT algorithm.

Machine learning algorithms trained using routinely collected data outperformed traditional statistical models when predicting 90-day ACU. The XGBT algorithm has the potential to identify high-ACU risk patients and enable preventive interventions to avoid ACU.