错误分类和异常值对归算方法的影响
1Institute for Competitiveness and Communication, School of Business, University of Applied Sciences and Art Northwestern Switzerland, Olten, Switzerland.
Journal of applied statistics
|October 25, 2024
概括
强大的条件归算方法在数据假设被违反时表现出色,在现实场景和模拟中表现优于其他技术. 这项研究评估了非理想条件下的归算性能,包括异常值和错误分类.
科学领域:
- 统计 统计 统计 统计
- 数据科学数据科学数据科学
- 机器学习 机器学习
背景情况:
- 存在许多归算方法,但主要是根据理想数据假设进行评估.
- 现实世界的数据往往偏离了理想的假设,像异常值,错误分类和模型错误规范等问题很常见.
- 在这些非理想条件下计算技术的性能尚未得到充分研究.
研究的目的:
- 调查各种归算方法对违反数据和模型假设的易感性.
- 评估归算技术在异常值,错误分类和不正确的模型规格的情况下的表现.
- 用模拟和现实数据比较不同归算方法的有效性.
主要方法:
- 在非理想条件下对归算方法的评估,特别是针对异常值,错误分类和模型错误规范.
- 使用各种评估指标进行归算技术的比较,包括归算值与真值的比较,统计比较,分类器性能和参数方差.
- 模拟研究和对现实世界数据集的分析,以测试归算方法的稳定性.
主要成果:
- 推算方法显示不同程度的易受违反理想化的假设.
- 异常值和错误分类显著降低了大多数标准归算方法的性能.
- 强大的条件归算方法在模拟和现实环境中与不完美的数据相比,表现优于其他方法.
结论:
- 当应用于违反理想化的假设的真实世界数据时,标准归算方法可能不可靠.
- 强大的条件归算为在数据质量和模型假设受到损害的实际应用中处理缺失数据提供了更可靠的方法.
- 对于在非理想环境中可靠的数据分析,对强大的归算技术进行进一步的研究是有必要的.
更多相关视频
06:55Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
14.4K
07:35Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
Published on: October 11, 2018
7.4K
相关概念视频
Outliers and Influential Points
4.0K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.0K
Detection of Gross Error: The Q Test
5.6K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
5.6K
What Are Outliers?
3.6K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.6K
Quantifying and Rejecting Outliers: The Grubbs Test
1.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.5K
Modified Boxplots
9.1K
A standard box and whisker plot informs us about the spread of the data in a given sample. One can identify the minimum value, maximum value, first quartile value, second quartile or median value, and third quartile.
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
9.1K
Survival Tree
61
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
61
