Impact on bias mitigation algorithms to variations in inferred sensitive attribute uncertainty.

Area of Science:

• Artificial Intelligence
• Machine Learning Ethics
• Algorithmic Fairness

Background:

• Growing concerns exist regarding the trustworthiness, fairness, and privacy of AI systems.
• Bias mitigation algorithms often require sensitive attribute data, which is increasingly unavailable.
• Inferring missing sensitive attributes offers a potential solution for applying bias mitigation.

Purpose of the Study:

• To investigate the robustness of bias mitigation algorithms against varying levels of inferred sensitive attribute accuracy.
• To assess the impact of inference accuracy on the performance of different bias mitigation strategies.
• To determine if bias mitigation can improve fairness in AI systems with inferred sensitive attributes.

Main Methods:

• Generated variations in sensitive attribute accuracy through simulation and neural model construction.
• Evaluated six bias mitigation algorithms across pre-processing, in-processing, and post-processing stages.
• Assessed fairness scores and balanced accuracy compared to a standard model.

Main Results:

• The disparate impact remover demonstrated the least sensitivity to inference accuracy.
• Bias mitigation using reasonably accurate inferred attributes yielded higher fairness scores than the standard model.
• Balanced accuracy remained comparable to the standard model when using inferred attributes.

Conclusions:

• Bias mitigation strategies can effectively improve AI fairness even when using inferred sensitive attributes.
• Reasonable inference accuracy is sufficient to achieve fairness gains without significant performance loss.
• This approach offers a pathway to enhance fairness in black box AI systems.