使用受限制分布转换的半监督异常检测
概括
这项研究引入了一种新的半监督异常检测 (AD) 方法,该方法有效地利用有限的标记异常数据. 该方法生成中间样本,以提高复杂场景中的检测性能.
科学领域:
- 计算机科学 计算机科学
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 异常检测 (AD) 通常被视为无监督的任务,当有限的标记异常数据可用时,缺乏有效的方法.
- 现有的半监督的AD方法无法充分利用稀缺的异常样本,减少它们对检测的影响.
- 诸如故障诊断和疾病检测等现实世界的应用程序经常呈现一些标记异常的场景.
研究的目的:
- 开发一种新的半监督异常检测 (SAD) 方法,最大限度地利用有限的标记异常数据.
- 通过有效地将稀缺的异常样本集成到学习过程中,提高异常检测的性能.
- 解决当前SAD技术在充分利用现有的专家标记异常数据方面的局限性.
主要方法:
- 提出了一种新的SAD方法,学习非线性转换以将正常和异常数据映射成不同的,不重叠的目标分布.
- 为了克服异常样本的稀缺性,通过正常和异常数据之间的插值生成中间样本.
- 这些中间样本被投射到位于正常分布和异常分布之间的第三个目标分布中.
主要成果:
- 与现有的监督和半监督AD方法相比,提出的SAD方法显示出更高的性能.
- 跨多个基准和多个领域的经验结果验证了该方法的有效性.
- 该方法通过充分利用有限的标记异常数据,成功地提高了检测性能.
结论:
- 开发的SAD方法通过有效处理有限的标记异常数据,在异常检测方面取得了重大进展.
- 生成和预测中间样本的策略对于改善SAD性能至关重要.
- 这项研究为在缺乏标记异常数据的场景中检测异常提供了更强大,更有效的解决方案.
相关概念视频
The Anderson-Darling Test
883
The Anderson-Darling test is a statistical method used to determine whether a data sample is likely drawn from a specific theoretical distribution. Unlike parametric tests, it does not require assumptions about specific parameters of the distribution. Instead, it compares the sample's empirical cumulative distribution function (ECDF) with the cumulative distribution function (CDF) of the hypothesized distribution. Critical values for the test are specific to the chosen distribution rather...
883
Unusual Results
3.3K
Unusual results are those that have a very low chance of occurring. Unusual results can be identified using probabilities and the range rule of thumb. In problems involving probability, unusual results can be observed in 2 instances – an unusually high number of successes or an unusually low number of successes.
According to the range rule of thumb, any value above or below two standard deviations, 2σ from the mean, μ is considered unusual.
Maximum unusual value =...
According to the range rule of thumb, any value above or below two standard deviations, 2σ from the mean, μ is considered unusual.
Maximum unusual value =...
3.3K
Student t Distribution
6.5K
The population standard deviation is rarely known in many day-to-day examples of statistics. When the sample sizes are large, it is easy to estimate the population standard deviation using a confidence interval, which provides results close enough to the original value. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
The Student t distribution was developed by William S. Goset (1876–1937) of the...
The Student t distribution was developed by William S. Goset (1876–1937) of the...
6.5K
Quantifying and Rejecting Outliers: The Grubbs Test
2.1K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.1K
Distributions to Estimate Population Parameter
4.3K
The accurate values of population parameters such as population proportion, population mean, and population standard deviation (or variance) are usually unknown. These are fixed values that can only be estimated from the data collected from the samples. The estimates of each of these parameters are sample proportion, the sample mean, and sample standard deviation (or variance). To obtain the values of these sample statistics, data are required that have particular distribution and central...
4.3K
Survival Tree
166
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
166


