Related Experiment Videos
Machine Learning for Coronary Heart Disease Prediction: Comparative Analysis of Framingham and Cleveland Subset of
Shreyas Raman1, Devansh Thakkar1, Jacques Calixte2
1Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115, USA.
Insights
Machine learning models show promise for predicting ischemic heart disease risk, with performance varying by dataset. SHAP analysis identified key predictors like blood pressure and smoking status, enhancing clinical interpretability.
Area of Science:
- Cardiology and Artificial Intelligence
- Machine Learning in Healthcare
- Biomedical Data Science
Background:
- Cardiovascular disease (CVD) is a leading global cause of death, with ischemic heart disease (IHD) accounting for a significant portion.
- Machine learning (ML) and deep learning (DL) show potential for improving heart disease risk prediction and feature identification.
Purpose of the Study:
- To develop an AI/ML framework for predicting and classifying ischemic heart disease (IHD) risk.
- To utilize publicly available datasets (Framingham Heart Study, UCI Heart Disease dataset) for model development.
- To employ SHAP (SHapley Additive exPlanations) for model interpretability.
Main Methods:
- Implemented a leakage-controlled machine learning pipeline with data cleaning, stratified splitting, feature scaling, and class balancing.
- Evaluated multiple ML models including logistic regression, random forest, K-nearest neighbors, XGBoost, and a feedforward neural network.
- Assessed model performance using metrics like accuracy, AUROC, and Brier score, alongside SHAP for interpretability and Brier score for calibration.
Main Results:
- Model performance varied by dataset; logistic regression showed strong threshold-based performance on the Cleveland subset, while K-nearest neighbors achieved the highest AUROC.
- On the Framingham dataset, logistic regression yielded the highest AUROC, random forest the lowest Brier score, and the feedforward neural network the highest accuracy.
- SHAP analysis identified key predictors such as systolic blood pressure, smoking status, hypertension, number of major vessels, chest pain type, and thallium stress-test results.
Conclusions:
- Optimal ML model performance for IHD risk prediction is dataset-dependent.
- SHAP analysis significantly enhances the clinical interpretability of ML models in cardiology.
- Increased access to de-identified medical data is crucial for advancing reproducible ML research in cardiology.
Introduction:
Cardiovascular disease (CVD) remains the leading cause of mortality worldwide, with coronary artery disease (CAD), also known as ischemic heart disease (IHD), responsible for approximately 13% of global deaths in 2021. Studies applying machine learning (ML) and deep learning (DL) to heart disease classification have demonstrated promising results in risk prediction and feature extraction.
Background/Objectives:
In this study, we develop an AI/ML framework to predict and classify ischemic heart disease risk using publicly available datasets, the Framingham Heart Study and the Cleveland subset of the UCI Heart Disease dataset, along with explanations for how predictions were made by a process called SHAP (SHapley Additive exPlanations).
Methods:
We implemented a leakage-controlled machine learning pipeline that included data cleaning, stratified 80/20 train-test splitting, training-fold-only feature scaling and class balancing, 5-fold hyperparameter tuning, SHAP interpretability, and Brier score-based calibration assessment. Logistic regression, random forest, K-nearest neighbors, XGBoost, and a feedforward neural network were evaluated on the Framingham dataset and the Cleveland subset of the UCI Heart Disease dataset. Performance was assessed using accuracy, precision, recall, F1-score, Matthews correlation coefficient, AUROC, and Brier score.
Results:
After leakage-controlled evaluation, Framingham performance was more modest than in the preliminary analysis. Logistic regression achieved the highest AUROC on the Framingham dataset (0.7234), while random forest achieved the lowest Brier score (0.1750), and the feedforward neural network achieved the highest accuracy (0.7719). On the Cleveland subset, logistic regression achieved the strongest threshold-based performance (accuracy 0.8667, precision 0.8571, recall 0.8571, F1-score 0.8571, MCC 0.7321), whereas K-nearest neighbors achieved the highest AUROC (0.9531) and lowest Brier score (0.0942). SHAP highlighted systolic blood pressure, smoking status, and hypertension as influential predictors (Framingham) and number of major vessels, chest pain type, thallium stress-test result (thal; normal, fixed defect, or reversible defect), and age (Cleveland) as top predictors.
Conclusions:
Optimal model performance is dataset-dependent, and SHAP enhances clinical interpretability. Broader access to high-quality, de-identified medical data could accelerate reproducible ML research in cardiology.