在现实世界中使用分类准确度指标的挑战:从回忆和精度到马修斯相关系数
1School of Geography, University of Nottingham, Nottingham, Nottinghamshire, United Kingdom.
PloS one
|October 4, 2023
概括
包括马修斯相关系数 (MCC) 在内的分类准确度指标,由于流行率变化和不完美的参考标准,经常被错误估计. 这些问题可能导致对分类性能的误导性解释.
科学领域:
- * 计算机科学 计算机科学
- * 医学 * 医学
- * 环境科学 环境科学
背景情况:
- *准确的分类对于解释,使用和决策至关重要.
- * 显而易见的分类准确性往往与真实的准确性有很大差异.
- *流行率的变化和不完美的参考标准是误估的主要原因.
研究的目的:
- *重新检查流行率变化和参考标准质量对二进制分类准确度指标的影响.
- * 具体分析马修斯相关系数 (MCC) 和其所声称的平衡绩效评估属性.
- * 调查流行指标如何受到不完美的数据的影响.
主要方法:
- *重新审视二进制分类中流行率和参考标准质量的基本问题.
- *分析流行率和参考标准缺陷对各种准确度指标的影响.
- * 进行具有现实的数据质量值的模拟,例如遥感中的模拟.
主要成果:
- * 流行准确度指标 (回忆,精度,特异性,NPV,J,F1,概率比率,MCC) 和患病率受到患病率变化和参考标准质量的影响.
- *模拟表明,在流行范围和参考标准质量水平范围内,指标值存在差异.
- * 马修斯相关系数 (MCC) 可能被大大低估或高估,而明显高的MCC可能是由于不良分类的结果.
结论:
- * 马修斯相关系数 (MCC) 和其他准确度指标的实用性可能被夸大了.
- * 表面准确度和流行率值可能会误导.
- *有必要认识到并解决流行率和参考标准质量对分类准确性评估的影响.
相关概念视频
Statistical Analysis: Overview
6.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.6K
Uncertainty in Measurement: Accuracy and Precision
73.8K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
73.8K
Accuracy, limits, and approximation
459
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
459
Receiver Operating Characteristic Plot
269
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
269
Accuracy and Errors in Hypothesis Testing
207
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
207
Random and Systematic Errors
11.0K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
11.0K


