Bias-inducing geometries: An exactly solvable data model with fairness implications

Machine learning (ML) may be oblivious to human bias but it is not immune to its perpetuation. Marginalization and iniquitous group representation are often traceable in the very data used for training and may be reflected or even enhanced by the learning models. In the present work, we aim to clarify the role played by data geometry in the emergence of ML bias. We introduce an exactly solvable high-dimensional model of data imbalance, where parametric control over the many bias-inducing factors allows for an extensive exploration of the bias inheritance mechanism. Through the tools of statistical physics, we analytically characterize the typical properties of learning models trained in this synthetic framework and obtain exact predictions for the observables that are commonly employed for fairness assessment. Simplifying the nature of the problem to its minimal components, we can retrace and unpack typical unfairness behavior observed on real-world datasets. Finally, we focus on the effectiveness of bias mitigation strategies, first by considering a loss-reweighing scheme that allows for an implicit minimization of different unfairness metrics and a quantification of the incompatibilities between existing fairness criteria. Then, we propose a mitigation strategy based on a matched inference setting that entails the introduction of coupled learning models. Our theoretical analysis of this approach shows that the coupled strategy can strike superior fairness-accuracy trade-offs.