A Responsible Framework for Assessing, Selecting, and Explaining Machine Learning Models in Cardiovascular Disease...

Area of Science:

• Machine Learning in Healthcare
• Clinical Predictive Modeling
• Health Informatics

Background:

• Trustworthy machine learning (ML) in clinical practice requires interpretability, explainability, and fairness.
• Interpretability alone does not guarantee explainability, which provides insights into model predictions.
• Current ML evaluation often prioritizes accuracy over broader trustworthiness aspects.

Purpose of the Study:

• To propose a 3-stage ML framework for responsible model development: assessment, selection, and explanation.
• To apply this framework for predicting cardiovascular disease (CVD) outcomes (myocardial infarction and stroke) in type 2 diabetes (T2D) patients.
• To evaluate the trade-offs between predictive accuracy and fairness in ML models.

Main Methods:

• Utilized the ACCORD dataset (N=9635) of T2D patients, including demographic, clinical, and biomarker data.
• Developed interpretable ML models (linear, tree-based, ensemble) using hold-out cross-validation.
• Assessed models using accuracy (AUC) and fairness (RPPS) metrics, quantifying trade-offs and employing SHAP and partial dependence plots for explainability.

Main Results:

• The GLMnet model demonstrated the best balance of performance and fairness for predicting myocardial infarction (MI) and stroke.
• GLMnet achieved high Relative Parity of Performance Scores (RPPS) for gender and race, indicating minimal disparities.
• Key predictors identified: history of CVD and age for MI; HbA1c and systolic blood pressure for stroke.

Conclusions:

• Established a responsible framework for ML model assessment, selection, and explanation, focusing on accuracy-fairness trade-offs.
• Highlighted that simpler models can match complex ensembles, but accuracy disparities across groups require attention.
• Emphasized the need for holistic approaches integrating accuracy, fairness, and explainability for enhanced healthcare technology adoption.