在有噪音标签的学习中,用于样本选择的异常修剪的双间隔光滑损失
Senyu Hou1, Maolong Xu1, Gaoxia Jiang1
1School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi, 030006, China.
概括
这项研究引入了异常修剪双间隔平滑 (OTDIS) 损失,以提高深度神经网络 (DNN) 在带有噪音标签的数据集上的性能. OTDIS提高了样本选择的准确性,减少了过拟合,提高了分类性能.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 计算机视觉 计算机视觉
背景情况:
- 现实世界数据集经常含有噪音标签,由于参数复杂度高,可能会导致深度神经网络 (DNN) 过度匹配.
- 用噪音标签学习 (LNL) 的样本选择方法通过损失值识别干净和噪音样本,但时间不稳定会导致选择错误.
- 错误地排除干净样本和保留噪音实例,阻碍了NLNL传统样本选择的有效性.
研究的目的:
- 提出一种新的异常调整双间隔平滑 (OTDIS) 损失函数,以提高样本选择的稳定性并减轻DNN过度适应标签噪声.
- 在代优化过程中解决损失不稳定性,通过将时间动态和样本分布集成的双间隔估计重新定义噪声水平.
- 通过调查早期培训阶段异常值损失的影响,提高样本选择的可靠性.
主要方法:
- 使用异常切割的置信区间下界实现时间平滑,以提高样本选择的时间稳定性.
- 通过基于集群的重新分组来利用样本空间平滑,以获得分布稳定的损失估计.
- 制定一个双极性培训目标,将负损失作为惩罚,并建立基于OTDIS损失的共同和半监督的学习框架.
主要成果:
- 与现有方法相比,OTDIS损失显著提高了样本选择的准确性.
- 在使用合成噪声的基准数据集 (MNIST,CIFAR) 上实现了优异的分类性能.
- 在现实世界杂数据集上表现出有效性,包括CIFAR-N,ANIMAL-10N和WebVision.
结论:
- 通过稳定样本选择和减少过度拟合,OTDIS损失为学习与噪音标签 (LNL) 提供了强大的解决方案.
- 拟议的方法提高了各种数据集的DNN性能,包括那些具有现实世界标签噪声的数据集.
- OTDIS为提高在不完美的数据上训练的机器学习模型的可靠性和准确性提供了宝贵的工具.
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
2.1K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.1K
Trimmed Mean
3.0K
While measuring the mean of a data set, care needs to be taken when associating the mean to its central tendency. The same goes for the arithmetic mean, the geometric mean, or the harmonic mean. This is because the presence of a single outlier data value can significantly affect the mean. That is, the mean is sensitive to fluctuations in the data set.
Although certain measures of central tendency are not sensitive to outliers, there are alternative versions of the mean that get around the...
Although certain measures of central tendency are not sensitive to outliers, there are alternative versions of the mean that get around the...
3.0K
Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
Truncation in Survival Analysis
318
Truncation in survival analysis refers to the exclusion of individuals or events from the dataset based on specific criteria related to the time of the event. This exclusion can happen in two primary forms: left truncation and right truncation.
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
318
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
What Are Outliers?
4.2K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
4.2K


