Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health...

Area of Science:

• Medical Informatics
• Clinical Prediction Models
• Surgical Outcomes

Background:

• Machine learning (ML) shows promise for predicting postoperative delirium.
• Real-world advantages and comparisons with conventional models are unclear.

Purpose of the Study:

• Validate temporal generalizability of ML models (XGBoost, LASSO) versus logistic regression for predicting postoperative delirium.
• Compare predictive performance in practical surgical settings.

Main Methods:

• Utilized electronic health records from 11,863 surgical patients (Dec 2017-Feb 2022).
• Developed XGBoost (decision tree ensemble) and LASSO (sparse linear regression) models.
• Compared models using AUROC, MCC, calibration slopes/intercepts, and Brier scores, with cohorts split pre- and during COVID-19 pandemic.

Main Results:

• XGBoost and LASSO models showed similar predictive discrimination (AUROC 0.86-0.90).
• A logistic regression model with 8 predictors (age, ICU, neurosurgery, etc.) demonstrated good performance (AUROC 0.84-0.88).
• No significant performance differences were found between ML and logistic models.

Conclusions:

• XGBoost and LASSO models did not significantly outperform each other or logistic regression in predicting postoperative delirium.
• A parsimonious logistic model with key predictors achieves comparable performance to complex ML models.