对检测异常值的相对范围的经验评估
Dania Dallah1, Hana Sulieman1, Ayman Al Zaatreh1
1Department of Mathematics and Statistics, American University of Sharjah, Sharjah 26666, United Arab Emirates.
Entropy (Basel, Switzerland)
|July 29, 2025
概括
本研究介绍了一种简单的范围分布方法,用于在单变量数据中有效检测异常值. 适应性方法在各种数据分布和条件中具有强大的性能,提高了数据质量.
科学领域:
- 数据科学数据科学数据科学
- 统计分析 统计分析
- 机器学习 机器学习
背景情况:
- 检测异常值对于数据质量和识别欺诈等异常至关重要.
- 现有的方法可能是复杂的或低效的,这取决于数据的分布.
- 检测单变异异常值需要强大的和可适应的统计方法.
研究的目的:
- 探索一种简单而有效的范围分布方法,用于单变异异常值检测.
- 为了比较标准偏差 (σ) 和四分位数间范围 (IQR) 规范化的范围统计的性能.
- 确定在各种分布中识别异常值的最佳值.
主要方法:
- 使用正常化范围统计 (范围/σ和范围/IQR) 检测调查的异常值.
- 在正常分布,物流分布,拉普拉斯分布和韦布尔分布中评估性能.
- 在不同的污染策略下,用不同的样本大小 (n) 和错误率 (α=0.1,0.05,0.01) 进行测试.
主要成果:
- 提出的基于范围的统计数据显示出异常值检测的灵活性和稳定性.
- 在不同的数据分布和样本大小中,性能是一致的.
- 确定了可靠的异常识别的有效值.
结论:
- 一种简单的适应性范围分布方法为单变量数据提供可靠的异常值检测.
- 该方法适用于各种数据环境,并保持检测准确度.
- 这种方法通过提高数据质量和发现不寻常的模式来增强数据分析.
更多相关视频
06:02Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios
Published on: October 6, 2020
2.4K
07:28Psychophysically-anchored, Robust Thresholding in Studying Pain-related Lateralization of Oscillatory Prestimulus Activity
Published on: January 21, 2017
7.1K
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
2.1K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.1K
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
What Are Outliers?
4.2K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
4.2K
Outliers and Influential Points
4.2K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.2K
Range Rule of Thumb to Interpret Standard Deviation
9.3K
The range rule of thumb in statistics helps us calculate a dataset's minimum and maximum values with known standard deviation. This rule is based on the concept that 95% of all values in a dataset lie within two standard deviations from the mean.
For instance, the range rule of thumb can be used to find the tallest and the shortest student in a class, given the mean student height and standard deviation. If the mean student height is 1.6 m and the standard deviation, s is 0.05 m, the height...
For instance, the range rule of thumb can be used to find the tallest and the shortest student in a class, given the mean student height and standard deviation. If the mean student height is 1.6 m and the standard deviation, s is 0.05 m, the height...
9.3K
Range
12.0K
The range is one of the measures of variation. It can be defined as the difference between a dataset's highest and lowest values. For example, in the study of seven 16-ounce soda cans, the filled volume of soda was measured, thus producing the following amount (in ounces) of soda:
15.9; 16.1; 15.2; 14.8; 15.8; 15.9; 16.0; 15.5
Measurements of the amount of soda in a 16-ounce can vary since different subjects record these measurements or since the exact amount - 16 ounces of liquid, was not...
15.9; 16.1; 15.2; 14.8; 15.8; 15.9; 16.0; 15.5
Measurements of the amount of soda in a 16-ounce can vary since different subjects record these measurements or since the exact amount - 16 ounces of liquid, was not...
12.0K
