An intentional approach to managing bias in general purpose embedding models

Area of Science:

• Machine Learning in Healthcare
• Artificial Intelligence Ethics
• Medical Image Analysis

Background:

• Machine learning in healthcare raises concerns about bias and exacerbating care disparities.
• Medical images can contain sensitive attribute signals, posing challenges for algorithmic fairness.
• Designing bias-free general purpose pretrained embeddings (GPPEs) for healthcare is complex.

Purpose of the Study:

• To investigate optimal design strategies for GPPEs to mitigate bias in downstream healthcare models.
• To analyze the unintended consequences of removing sensitive attributes from GPPEs.
• To emphasize the importance of rigorous evaluation and diverse teams in developing fair AI models.

Main Methods:

• Review and synthesis of previously published data on machine learning bias in healthcare.
• Conceptual analysis of GPPE design and its impact on downstream model performance.
• Argumentation for retaining information in GPPEs and perils of attribute removal.

Main Results:

• Attempts to remove sensitive attributes from GPPEs can lead to unintended bias and poor performance in downstream models.
• GPPEs should ideally retain as much information as the original data.
• Technical neutrality in upstream components does not guarantee an unbiased end-to-end system.

Conclusions:

• Prioritize retaining information in GPPEs rather than removing sensitive attributes.
• Downstream models must be carefully designed, evaluated for bias, and audited.
• Diverse teams and diverse patient cohorts are crucial for developing equitable healthcare AI.