关于对异常值检测和一类分类的评估:对算法进行比较研究,选择模型和集合
Henrique O Marques1, Lorne Swersky2, Jörg Sander2
1University of Southern Denmark, Odense, Denmark.
概括
本研究比较了一类分类和无监督异常值检测方法. 支持向量数据描述 (SVDD) 和高斯混合模型 (GMM) 通常表现最好,即使没有标记异常值数据.
科学领域:
- 机器学习 机器学习
- 数据挖掘 数据挖掘
- 模式识别 模式识别
背景情况:
- 无监督异常值检测方法可以适应一个类别的分类.
- 以前的比较有局限性,特别是在没有标记异常值的模型选择中.
研究的目的:
- 严格比较一类分类算法与适应的无监督异常值检测方法.
- 在没有标记异常值时评估模型选择策略.
主要方法:
- 实验性比较各种单类分类和异常值检测算法.
- 对具有不同特征的不同数据集进行评估.
- 使用多个指标评估绩效.
- 模型选择方法的比较,有和没有基准真理异常值数据.
主要成果:
- 支持向量数据描述 (SVDD) 和高斯混合模型 (GMM) 显示出强大的整体性能.
- 性能是一致的,无论地面真相数据是否用于参数选择.
- 特定的方法可能在特定的应用场景中表现出色.
- 一个类分类器的集合可以提高准确性,如果成员被精心选择.
结论:
- 对于一类分类任务,SVDD和GMM是强大的选择.
- 没有标记异常值的模型选择是可行的和有效的.
- 集合方法为提高分类准确性提供了一个有希望的途径.
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
1.7K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.7K
Outliers and Influential Points
4.1K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.1K
Detection of Gross Error: The Q Test
6.3K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.3K
What Are Outliers?
3.9K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.9K
Comparing the Survival Analysis of Two or More Groups
226
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
226
Sensitivity, Specificity, and Predicted Value
499
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
499


