What drives performance in machine learning models for predicting heart failure outcome?

Area of Science:

• Cardiology
• Medical Informatics
• Machine Learning

Background:

• Acute heart failure (AHF) presents a critical juncture with poor prognosis.
• Current risk-stratification tools at hospital discharge are inadequate for tailored treatment.
• Machine learning offers potential for improved AHF risk prediction using complex patient data.

Purpose of the Study:

• To identify key factors driving success in AHF prediction models.
• To develop an AI-based prediction model tailored to a specific institution for real-time clinical decision support.

Main Methods:

• A cohort of 10,868 AHF patients over 12 years was analyzed.
• 372 covariates were collected from admission through hospitalization.
• Seven machine learning models were evaluated, including logistic regression, random forest, Cox, XGBoost, NeuralNet, and an ensemble model.
• Model performance was assessed based on prediction method and covariate type/number, with 1-year survival as the primary outcome.

Main Results:

• Most models achieved >80% prediction accuracy (AUROC).
• The ensemble model showed slightly superior performance (81.2% AUROC).
• The number and type of covariates significantly impacted prediction success (P < 0.001), with multiplex-covariates outperforming traditional clinical variables (80.4% vs. 77.8% AUROC).
• Demographics, lab tests, and administrative data provided the most significant performance gains.

Conclusions:

• The selection of predictive modeling method is less critical than the multiplicity and type of covariates used for AHF prognosis.
• Structured data preprocessing and the use of multiple covariates yield accurate, institution-specific risk predictions for AHF.