Muffin: A Framework Toward Multi-Dimension AI Fairness by Uniting Off-the-Shelf Models

Area of Science:

• Artificial Intelligence
• Machine Learning Ethics
• Algorithmic Fairness

Background:

• Model fairness, or bias, is a critical issue in AI applications like autonomous driving and healthcare.
• Existing research primarily addresses single unfair attributes, neglecting multi-dimensional fairness in real-world data with multiple attributes.
• A strong correlation often exists between different unfair attributes, where optimizing one can negatively impact others.

Purpose of the Study:

• To investigate the challenges of multi-dimensional fairness in AI models.
• To propose a novel framework, Muffin, for simultaneously improving fairness across multiple attributes.
• To develop an automatic tool within Muffin to unite existing models for enhanced multi-attribute fairness.

Main Methods:

• Analysis of correlations between different unfair attributes in multi-dimensional datasets.
• Development of the Muffin framework, an automatic tool for uniting off-the-shelf models.
• Case studies on dermatology datasets to evaluate Muffin's performance against existing approaches.

Main Results:

• Existing methods show limited fairness improvement on one attribute while causing unfairness on another (e.g., 21.05% on attribute 1, -1.85% on attribute 2).
• The proposed Muffin framework achieved simultaneous fairness improvements of 26.32% and 20.37% on two attributes.
• Muffin also resulted in a 5.58% gain in model accuracy.

Conclusions:

• Optimizing fairness for a single attribute can lead to the collapse of fairness for other attributes.
• The Muffin framework effectively addresses multi-dimensional fairness by uniting models to improve fairness across multiple attributes concurrently.
• Muffin offers a promising solution for developing more equitable AI systems in diverse real-world applications.