Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction

Area of Science:

• Cardiovascular disease risk prediction
• Machine learning in healthcare
• Health equity and bias analysis

Background:

• Systematic bias in health data can compromise the performance of machine learning (ML) models for cardiovascular disease (CVD) risk assessment.
• Ensuring equitable performance across demographic groups is crucial for reliable CVD risk prediction.
• Investigating bias in ML models and evaluating debiasing strategies is essential for fair healthcare applications.

Purpose of the Study:

• To evaluate the performance equivalence of ML-based CVD risk prediction models across different demographic groups (race and gender).
• To assess the effectiveness of bias mitigation methods in reducing disparities within these ML models.
• To compare the fairness of ML models against a traditional baseline model, the AHA Pooled Cohort Risk Equations (PCEs).

Main Methods:

• Utilized de-identified Electronic Health Records (EHR) data from Vanderbilt University Medical Center.
• Applied various ML algorithms (logistic regression, random forest, gradient-boosting trees, LSTM) to develop predictive models.
• Assessed model bias and fairness using Equal Opportunity Difference (EOD) and Disparate Impact (DI), comparing ML models with AHA PCEs and testing three debiasing techniques.

Main Results:

• Most ML models demonstrated lower bias (EOD, DI) compared to AHA PCEs across race groups.
• Significant bias was observed across gender groups for both ML models and AHA PCEs.
• Resampling by case proportion effectively reduced gender-based bias but showed limited impact on race-based bias and sometimes decreased model accuracy.

Conclusions:

• Both traditional and ML-based CVD risk prediction models exhibit bias against women, highlighting the need for targeted interventions.
• Bias mitigation strategies, particularly resampling by proportion, show promise for addressing gender disparities in CVD risk prediction.
• Further research is needed to develop effective debiasing methods that do not compromise predictive accuracy and address all demographic groups equitably.