权重低级近似的应用:在数据矩阵中检测异常值
Marisol García-Peña1, Sergio Arciniegas-Alarcón2, Kaye E Basford3
1Departamento de Matemáticas, Pontificia Universidad Javeriana, Bogotá, Colombia. marisolgarcia@javeriana.edu.co.
BMC research notes
|May 26, 2025
概括
本研究引入了加权矩阵近似,用于在矩形数据集中有效检测异常值. 这些方法优于传统的偏差调整盒图,用于识别数字数据中的异常.
科学领域:
- 数据科学数据科学数据科学
- 统计分析 统计分析
- 生物信息学是一种生物信息学.
背景情况:
- 异常值的识别对于探索性数据分析至关重要.
- 异常值的存在会影响后续的建模选择.
- 现有的方法可能不适合所有数据结构.
研究的目的:
- 通过加权矩阵近似来提出新型策略来识别异常值.
- 评估这些策略在各种现实世界数据集上的有效性.
- 提出一个评估异常值检测性能的统计数据.
主要方法:
- 利用矩阵的加权近似来确定异常值.
- 评估了六个标准,包括残留物和杰克刀方法.
- 与偏差调整的盒子图表黄金标准进行性能比较.
- 在16个真实数据集上进行了测试,有人工污染.
主要成果:
- 与偏差调整盒图相比,权重近似方法在检测随机异常值方面表现出更高的有效性.
- 所有提出的方法都适用于任何矩阵形式的数值数据集.
- 拟议的评估统计有效地区分了良好的检测与假阳性/假阴性.
结论:
- 权重矩阵近似为在数值数据集中检测异常值提供了更有效的方法.
- 这些方法具有多功能性,适用于复杂的数据,包括基因型与环境相互作用.
- 该研究为评估异常值检测技术提供了一个强大的框架.
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
2.2K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.2K
Outliers and Influential Points
4.3K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.3K
What Are Outliers?
4.2K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
4.2K
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
Weighted Mean
5.4K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
5.4K
Wilcoxon Rank-Sum Test
355
The Wilcoxon rank-sum test, also known as the Mann-Whitney U test, is a nonparametric test used to determine if there is a significant difference between the distributions of two independent samples. This test is designed specifically for two independent populations and has the following key requirements:
355


