规范化批量规范化用于长尾识别
概括
本研究引入了一种新的方法,通过规范化批量规范化 (BN) 层参数来解决人工智能中不平衡的训练数据. 这种技术有效地平衡了特征表示,显著提高了长尾分布中罕见类的性能.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 现实世界数据集经常表现出长尾分布,其中一些类比其他类具有显著更多的样本.
- 传统的深度学习模型与不平衡的数据作斗争,导致代表性不足 (罕见) 类的表现不佳.
- 现有的方法主要集中在数据层面或分类器层面的调整,以减轻偏差.
研究的目的:
- 在经过不平衡数据训练的深度网络中调查特征级别偏差.
- 提出一种通过修改批量规范化 (BN) 层参数来纠正特征偏差的新方法.
- 在长尾识别任务中增强罕见类的模型性能.
主要方法:
- 确定对频繁类的偏见是编码在网络特征中,削弱罕见类的特定特征.
- 为批量规范化 (BN) 层引入了一个参数规范化技术.
- 代表了BN层重量/偏差参数作为向量,将它们规范化为单位向量,并乘以可学习的标量数,解离参数方向和大小.
主要成果:
- 提出的方法有效地将BN层参数规范化,从而实现更平衡的特征表示.
- 实验表明,在各种长尾基准中,稀有类的性能显著改善.
- 该方法在CIFAR-10/100-LT,ImageNet-LT和iNaturalist 2018等数据集上表现优于现有的最先进方法.
结论:
- 编码在网络参数中的特征偏差是罕见类表现差的一个关键因素.
- 规范化批量规范化 (BN) 层参数提供了一个简单而有效的解决方案来纠正特征偏差.
- 拟议的方法为提高深度学习模型在不平衡数据场景中的稳定性提供了一个有希望的方向.
相关概念视频
Applications of Normal Distribution
4.9K
The normal distribution is a useful statistical tool. One of its practical applications is determining the door height after considering the normal distribution of heights of persons, such that many can pass through it easily without striking their heads. The normal distribution can also determine the probability of a person having a height less than a specific height.
The heights of 15 to 18-year-old males from Chile from 1984 to 1985 followed a normal distribution. The mean height is 172.36...
The heights of 15 to 18-year-old males from Chile from 1984 to 1985 followed a normal distribution. The mean height is 172.36...
4.9K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Normal Distribution
10.5K
The normal, a continuous distribution, is the most important of all the distributions. Its graph is a bell-shaped symmetrical curve, which is observed in almost all disciplines. Some of these include psychology, business, economics, the sciences, nursing, and, of course, mathematics. Some instructors may use the normal distribution to help determine students’ grades. Most IQ scores are normally distributed. Often real-estate prices fit a normal distribution. The normal distribution is...
10.5K
Central Limit Theorem
14.3K
The central limit theorem, abbreviated as clt, is one of the most powerful and useful ideas in all of statistics. The central limit theorem for sample means says that if you repeatedly draw samples of a given size and calculate their means, and create a histogram of those means, then the resulting histogram will tend to have an approximate normal bell shape. In other words, as sample sizes increase, the distribution of means follows the normal distribution more closely.
The sample size, n, that...
The sample size, n, that...
14.3K
The Bell Curve
84
The normal probability distribution, often depicted as a symmetrical, bell-shaped curve, is fundamental in statistics and the study of natural phenomena. This pattern, famously described by mathematician Carl Friedrich Gauss, shows how data points are distributed around a central mean, with most values near the average and fewer observations occurring as they deviate further from it.
This pattern applies to many human characteristics beyond intelligence, such as height. For example, if you...
This pattern applies to many human characteristics beyond intelligence, such as height. For example, if you...
84
Unusual Results
3.1K
Unusual results are those that have a very low chance of occurring. Unusual results can be identified using probabilities and the range rule of thumb. In problems involving probability, unusual results can be observed in 2 instances – an unusually high number of successes or an unusually low number of successes.
According to the range rule of thumb, any value above or below two standard deviations, 2σ from the mean, μ is considered unusual.
Maximum unusual value =...
According to the range rule of thumb, any value above or below two standard deviations, 2σ from the mean, μ is considered unusual.
Maximum unusual value =...
3.1K


