Racial Equity in Healthcare Machine Learning: Illustrating Bias in Models With Minimal Bias Mitigation

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Health Equity

Background:

• Machine learning (ML) offers significant potential to improve healthcare outcomes.
• However, the risk of inherent inequities within ML algorithms requires careful evaluation.
• Assessing racial bias in healthcare ML is crucial for equitable application.

Purpose of the Study:

• To evaluate racial bias in five common ML algorithms used for healthcare access prediction.
• To determine if minimal bias mitigation strategies reduce racial disparities in model performance.
• To compare the performance of different ML algorithms in the presence of racial bias.

Main Methods:

• Utilized a CDC public database to construct ML models for predicting healthcare access.
• Employed Area Under the Curve (AUC) as the primary performance metric.
• Conducted 20 bootstrap analyses to generate confidence intervals for AUC metrics and compared race-specific performance.

Main Results:

• ML models demonstrated significantly higher performance for the White group compared to other racial groups across most algorithms.
• Extreme Gradient Boosting (XGBoost) was the most accurate algorithm, followed by random forest, naive Bayes, SVM, and KNN.
• Minimal bias mitigation strategies were insufficient to overcome existing racial performance disparities.

Conclusions:

• The study highlights the significant risk of racial disparities in healthcare ML models, even with limited bias mitigation.
• Existing ML models may perpetuate or worsen health inequities due to performance differences across racial groups.
• There is an urgent need for enhanced research, guidelines, and conversations on assessing and mitigating racial bias in healthcare ML.