Development, evaluation and validation of machine learning models to predict hospitalizations of patients with coronary artery disease within the next 12 months
Andrey D Ermak1, Denis V Gavrilov1, Roman E Novitskiy1
1K-SkAI LLC, Petrozavodsk, Russia.
Insights
A machine learning model effectively predicts hospitalizations in stable coronary artery disease (CAD) patients. This CatBoost model demonstrates high accuracy and stability, enabling better patient monitoring and care for those with CAD.
Area of Science:
- Cardiovascular Medicine
- Medical Informatics
- Machine Learning in Healthcare
Background:
- Rising life expectancy and improved survival post-acute coronary syndromes increase the prevalence of stable coronary artery disease (CAD).
- Stable CAD poses a significant healthcare burden due to disease progression, complications, and frequent hospitalizations.
- Predictive modeling for CAD hospitalizations can optimize care and resource allocation.
Purpose of the Study:
- To develop and externally validate personalized machine learning models for predicting preventable hospitalizations in stable CAD patients.
- To assess the efficacy of various ML algorithms using real-world clinical data.
Main Methods:
- Utilized 135,873 electronic health records from 49,103 patients with stable CAD.
- Evaluated 73 features including anthropometric, clinical, laboratory, and socio-demographic data.
- Compared logistic regression, discriminant analysis, naive Bayes, and tree-based methods (gradient boosting, bagging), with external validation on a separate regional dataset.
Main Results:
- The CatBoost model achieved the highest performance, with an AUC of 0.875 (internal) and 0.872 (external validation).
- The model demonstrated strong predictive power and stability on data with limited similarity to the training set.
- Performance was further confirmed by AUROC, Brier score, and standardized net benefit.
Conclusions:
- The developed CatBoost model surpasses previously published prediction models for CAD hospitalizations.
- External validation confirms the model's robustness and applicability in real-world clinical settings.
- This predictive tool supports proactive patient management and intervention for stable CAD.
Background:
Improved survival of patients after acute coronary syndromes, population growth, and overall life expectancy rise have led to a significant increase in the proportion of patients with stable coronary artery disease (CAD), creating a significant load on the entire healthcare system. The disease often progresses with the development of many complications while significantly increasing the likelihood of hospitalization. Developing and applying a machine learning model for predicting hospitalizations of patients with CAD to an inpatient medical facility will allow for close monitoring of high-risk patients, early preventive interventions, and optimized medical care.
Aims:
Development and external validation of personalized models for predicting the preventable hospitalizations of patients with stable CAD and its complications using ML algorithms and data of real-world clinical practice.
Methods:
135,873 depersonalized electronic health records of 49,103 patients with stable CAD were included in the study. Anthropometric measurements, physical examination results, laboratory, instrumental, anamnestic, and socio-demographic data, widely used in routine medical practice, were considered as potential predictors, a total of 73 features. Logistic regression, decision tree-based methods including gradient boosting (AdaBoost, LightGBM, XGBoost, CatBoost) and bagging (RandomForest and ExtraTrees), discriminant analysis (LinearDiscriminant, QuadraticDiscriminant), and naive Bayes classifier were compared. External validation was performed on the data of a separate region.
Results:
The best results and stability to external validation data were shown by the CatBoost model with an AUC of 0.875 (95% CI 0.865-0.885) for the internal testing and 0.872 (95% CI 0.856-0.886) for the external validation. The best model showed good performance evaluated through AUROC, Brier score and standardized net benefit (for the target NPV threshold) for the validation dataset that was only slightly similar to the train data.
Conclusion:
The metrics of the best model were superior to previously published studies. The results of external validation demonstrated the relative stability of the model to new data from another region that confirms the possibility of the model's application in real clinical practice.
