Misguided Artificial Intelligence: How Racial Bias is Built Into Clinical Models

Area of Science:

• Healthcare AI
• Medical Informatics
• Health Equity

Background:

• Artificial Intelligence (AI) offers solutions for numerous problems, including healthcare challenges.
• However, AI adoption in healthcare raises concerns about potentially exacerbating existing health inequities.
• Racial bias within AI systems is a significant barrier to equitable healthcare delivery.

Purpose of the Study:

• To investigate the presence and impact of racial bias in AI predictive models used in healthcare.
• To identify specific points within the AI model-building process where racial bias can be introduced.
• To explore the mechanisms through which racial bias manifests in healthcare AI.

Main Methods:

• Analysis of the AI model-building lifecycle, including data collection and preprocessing.
• Examination of data labeling practices and their susceptibility to bias.
• Review of AI model implementation strategies and their potential for disparate impact.

Main Results:

• Racial bias can infiltrate healthcare AI at multiple stages: raw data collection, data processing, and data labeling.
• Biased data and labeling can lead to predictive models that perpetuate or amplify existing health disparities.
• The implementation phase of AI models can also introduce or worsen racial bias if not carefully managed.

Conclusions:

• Addressing racial bias in healthcare AI is crucial for achieving health equity.
• Mitigation strategies must be applied throughout the AI development and deployment pipeline.
• Further research is needed to develop and validate methods for detecting and correcting racial bias in medical AI.