通常使用的软件工具产生冲突和过度乐观的AUPRC值
Wenyu Chen1, Chen Miao1, Zhenghao Zhang2
1School of Biomedical Sciences, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR, China.
bioRxiv : the preprint server for biology
|February 19, 2024
概括
精度回忆曲线 (AUPRC) 下的区域对于癌症诊断等不平衡的分类任务至关重要. 然而,流行的工具产生不同的AUPRC值,其中一些高估了分类器性能.
科学领域:
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
- 计算生物学 计算生物学
背景情况:
- 精度回忆曲线 (PRC) 和其下面的面积 (AUPRC) 是评估分类模型的关键指标,特别是在癌症诊断和细胞类型注释等领域常见的不平衡数据集.
- 这些指标在科学文献中得到广泛采用,有3000多项研究使用PRC绘图和AUPRC计算工具.
结论:
- 选择用于AUPRC计算的工具可以影响对分类性能的评估.
- 研究人员在解释AUPRC值时应谨慎行事,考虑到工具特定偏差的可能性.
- 需要对AUPRC计算进行进一步的标准化或明确的指导方针,以确保在科学研究中可靠的性能评估.
更多相关视频
16:23Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction
Published on: February 26, 2014
14.3K
11:53Spatial Multiobjective Optimization of Agricultural Conservation Practices using a SWAT Model and an Evolutionary Algorithm
Published on: December 9, 2012
13.0K
相关概念视频
01:25Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
01:10Propagation of Uncertainty from Systematic Error
519
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
519
00:59Uncertainty: Overview
555
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
555
01:11Introduction to R
268
R is a powerful software environment for statistical computing and graphics. Originating as an implementation of the S language, developed at Bell Laboratories, R has evolved into a robust, open-source statistical software favored by statisticians and data scientists worldwide. Its comprehensive suite includes data manipulation, calculation, and graphical display capabilities, making it versatile for data analysis and visualization. Its programming language is at the core of R's...
268
01:10Statgraphics
128
Statgraphics is a comprehensive statistical software suite designed for both basic and advanced data analysis. Originating in 1980 at Princeton University under Dr. Neil W. Polhemus, it was one of the pioneering tools for statistical computing on personal computers, with its public release in 1982 marking an early milestone in data science software. Over the years, it has evolved into a robust platform for data science, offering tools for regression analysis, ANOVA, multivariate statistics,...
128
00:59Propagation of Uncertainty from Random Error
684
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
684