Fairness-aware machine learning engineering: how far are we?

Area of Science:

• Software Engineering
• Artificial Intelligence
• Machine Learning Ethics

Background:

• Machine learning (ML) algorithms are increasingly integrated into daily life and business operations globally.
• Bias in ML algorithms can lead to unfair decision-making and perpetuate discrimination.
• There is a growing interest in software fairness within the software engineering community, yet understanding of fair ML engineering remains limited.

Purpose of the Study:

• To investigate the practical perception and management of fairness in ML systems.
• To identify knowledge gaps concerning practitioners' awareness, skills, and optimal development phases for addressing fairness.
• To offer insights into practical tools and approaches for effectively handling fairness in ML development.

Main Methods:

• A survey was conducted with 117 professionals involved in ML development.
• The survey gathered data on practitioners' experiences, awareness, and perceived challenges related to ML fairness.
• Analysis focused on understanding how fairness is currently managed in the software engineering lifecycle.

Main Results:

• Fairness is predominantly viewed as a secondary quality attribute in AI system development.
• Practitioners highlighted the need for specific methods, development environments, and automated validation tools to address fairness.
• There is a recognized gap in skills and maturity regarding the engineering of fair ML systems.

Conclusions:

• Fairness is not yet a primary consideration in the practical development of AI systems.
• Developing specialized tools and environments is crucial to integrate fairness considerations throughout the software lifecycle.
• Addressing fairness requires a shift towards treating it as a first-class quality aspect in ML engineering.