在链接预测中的评估指标之间存在不一致
Yilin Bi1, Xinshan Jiao1, Yan-Li Lee2
1CompleX Lab, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China.
PNAS nexus
|November 20, 2024
概括
选择正确的评估指标对于链接预测算法性能至关重要. 这项研究揭示了指标不一致性,并建议在网络科学中使用至少两个指标进行可靠的评估.
科学领域:
- 网络科学 网络科学
- 数据挖掘 数据挖掘
- 机器学习 机器学习
背景情况:
- 链接预测对于理解网络动态至关重要.
- 现有的链接预测算法评估指标经常被不一致地应用.
- 这种不一致性可能导致对算法性能进行不可靠的评估.
研究的目的:
- 调查不同评估指标对链接预测算法排名的影响.
- 确定可靠和全面的指标来评估链接预测性能.
- 建立一个标准,用于在网络科学中选择适当的评估指标.
主要方法:
- 在数百个现实世界的网络上进行了广泛的实验.
- 评估了26个众所周知的链接预测算法.
- 分析了各种评估指标产生的绩效排名.
主要成果:
- 在不同的评估指标中发现了显著的不一致性,导致了不同的算法排名.
- 没有一个单一的指标可以全面评估算法性能.
- 建议使用至少两个指标,如AUC和精度回忆,或AUC和规范化折扣累积收益.
结论:
- 评估指标的选择极大地影响了链接预测算法评估.
- 为了可信的绩效评估,需要多种指标的组合.
- 这项工作为开发用于链接预测指标选择的标准化标准提供了基础.
相关概念视频
Confidence Coefficient
7.5K
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under...
7.5K
Spearman's Rank Correlation Test
682
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates...
Spearman's test calculates...
682
Accuracy and Precision
8.7K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate...
8.7K
Multiple Comparison Tests
3.9K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.9K
Accuracy and Errors in Hypothesis Testing
176
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
176
Wilcoxon Signed-Ranks Test for Matched Pairs
87
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
87


