Development and Evaluation of Machine Learning Models to Predict the Risk of Major Cardiac Events and Death for...

People with kidney failure undergoing noncardiac surgery face an elevated risk of cardiovascular events and mortality. Existing risk prediction tools for perioperative events are either inaccurate in this population or include many variables that may complicate implementation. We developed and evaluated the performance of simplified machine-learning models for major cardiac events and mortality within 30 days after noncardiac surgery in patients with kidney failure in Alberta and Manitoba, Canada.

Data from Manitoba was split into training (70%), validation (15%), and testing (15%) sets. The training set was used for hyperparameter tuning and model training, the validation set for feature selection and evaluating model performance, and the testing set for final model performance. External evaluation was performed in a cohort from Alberta.

We included Manitoban adults (≥18 years) with kidney failure (eGFR < 15 mL/min/1.73 m² or on maintenance dialysis) undergoing noncardiac surgery (2007-2019), with evaluation data on adults from Alberta (2005-2019).

Variables included sex, age, surgery type and setting, kidney failure type, chronic conditions, and preoperative laboratory values (albumin and hemoglobin).

Composite of acute myocardial infarction, cardiac arrest, ventricular arrhythmia, and all-cause mortality within 30 days of surgery.

Model performance was evaluated using area under the receiver operating characteristic curve (AUC-ROC), area under the precision-recall curve (AUC-PR), and calibration. The final XGBoost and random forest models were externally validated using Alberta data.

Among 4,175 participants (12,082 surgeries), 569 outcomes (5%) were observed. The parsimonious XGBoost model (8 features) showed an AUC-ROC of 0.861 and AUC-PR of 0.304, and the parsimonious random forest model (19 features) estimated an AUC-ROC of 0.863 and AUC-PR of 0.332 in the testing cohort. External validation in Alberta showed similar performance with good calibration.

Lack of external validation outside Canada.

Our machine learning models were accurate and had improved parsimony over existing regression-based tools. Future work should test these models in other populations and compare them with regression-based models, in addition to assessing the value of these tools for informing risk-guided perioperative care.

People with kidney failure face high risks of cardiovascular problems and death after noncardiac surgery, but the current tools to predict these risks do not work well. This study developed new machine learning-based risk prediction models to help identify which patients are at higher risk after having noncardiac surgery. The models used a small number of easily available variables such as type of surgery, surgery setting, and laboratory results. We tested the models by using data from 2 Canadian provinces and found them to be accurate and reliable. These models may allow clinicians to inform patients of their individualized risk, which may support shared perioperative decision making. More research is needed to see how they perform in other geographic locations and health care systems.