在部署的机器学习算法中评估公平性的新方法
Shahadat Uddin1, Haohui Lu2, Ashfaqur Rahman3
1School of Project Management, Faculty of Engineering, The University of Sydney, Forest Lodge, Camperdown, NSW, 2037, Australia. shahadat.uddin@sydney.edu.au.
Scientific reports
|July 31, 2024
概括
本研究介绍了一种使用k倍交叉验证和t测试来评估机器学习 (ML) 公平性的统计验证方法. 结果表明,ML算法的公平性依赖于数据集,突出了适应性公平性定义的需要.
科学领域:
- 人工智能的人工智能
- 机器学习伦理学 机器学习伦理学
- 算法公平性 算法公平性
背景情况:
- 机器学习 (ML) 系统越来越多地影响社会各个部门,引发了对公平性的担忧.
- 现有的研究表明,在机器学习应用中普遍存在不公平的结果.
- 目前还没有统计验证的方法来评估部署的ML算法对数据集的公平性.
研究的目的:
- 引入一种新的,统计验证的方法来评估部署的ML算法的公平性.
- 通过使用既定的公平性定义,在基准数据集中评估经典ML算法的公平性.
- 调查ML公平性的上下文依赖性及其影响.
主要方法:
- 开发了一种使用k倍交叉验证和统计t测试的新型评估方法.
- 将方法应用于五个基准数据集和六个经典的ML算法.
- 从当前文献中考虑了四个不同的公平性定义.
主要成果:
- 同一个数据集对某些ML算法产生了公平的结果,对其他算法产生了不公平的结果.
- 公平性被证明是一个复杂的问题,高度依赖于特定的ML算法和数据集.
- 提出的方法成功地确定了不同ML模型中公平性结果的变化.
结论:
- 开发的方法使研究人员能够在数据集中的受保护属性上统计评估ML算法的公平性.
- 调查结果强调了需要适应性的公平性定义和特定环境的评估.
- 进一步研究提高公平的整体方法对于公平的AI部署至关重要.
相关概念视频
Bias
4.1K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
4.1K
Quantifying and Rejecting Outliers: The Grubbs Test
1.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.5K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
47
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
47
Friedman Two-way Analysis of Variance by Ranks
175
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
175
Randomized Experiments
6.8K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.8K
Bonferroni Test
2.7K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
2.7K


