对虚假关联的稳定性改善了语义分布外检测
Lily H Zhang1, Rajesh Ranganath1,2
1Center for Data Science, New York University.
概括
新的麻烦意识方法通过解决由共享的麻烦特征引起的故障来改善分布外 (OOD) 检测. 这种方法在现实场景中增强了异常检测,在现实场景中,OOD数据模仿了分布式数据.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 预测模型的输出和特征用于分布外 (OOD) 检测.
- 现有的方法无法检测OOD输入与分发数据共享麻烦特征 (例如背景).
- 分享麻烦的OOD (SN-OOD) 检测对于现实应用至关重要.
研究的目的:
- 解释SN-OOD检测中的故障.
- 提出一种新的麻烦意识的OOD检测方法.
- 提高OOD输入的检测与共享的麻烦功能.
主要方法:
- 将经验风险最小化 (ERM) 分类器替换为在破碎的骚扰-标签关系下训练的模型.
- 训练模型以产生无障碍的独立表示,无论是边缘的还是标签上的条件.
- 在训练中使用麻烦随机蒸 (NURD),这是一种在虚假相关性下对OOD概括的算法.
主要成果:
- 具有干扰意识的OOD检测显著优于原始方法.
- 基于输出和基于特征的麻烦意识方法都显示出实质性的改进.
- 提出的方法在域泛化算法未能提高性能的情况下取得成功.
结论:
- 干扰意识的OOD检测有效地解决了SN-OOD检测失败的问题.
- 该NURD算法使训练可用于强大的干扰独立表示.
- 这项工作为真实世界的图像输入提供了可靠的OOD检测的重大进步.
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
Detection of Gross Error: The Q Test
6.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.1K
Cause and Effect
10.9K
While variables are sometimes correlated because one does cause the other, it could also be that some other factor, a confounding variable, is actually causing the systematic movement in our variables of interest. For instance, as sales in ice cream increase, so does the overall rate of crime. Is it possible that indulging in your favorite flavor of ice cream could send you on a crime spree? Or, after committing crime do you think you might decide to treat yourself to a cone?
10.9K
Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
Types of Errors: Detection and Minimization
1.6K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
1.6K
Expected Frequencies in Goodness-of-Fit Tests
2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.5K


