Inconsistency among evaluation metrics in link prediction.

Area of Science:

• Network Science
• Data Mining
• Machine Learning

Background:

• Link prediction is vital for understanding network dynamics.
• Existing evaluation metrics for link prediction algorithms are often inconsistently applied.
• This inconsistency can lead to unreliable assessments of algorithm performance.

Purpose of the Study:

• To investigate the impact of different evaluation metrics on link prediction algorithm rankings.
• To identify reliable and comprehensive metrics for assessing link prediction performance.
• To establish a standard for selecting appropriate evaluation metrics in network science.

Main Methods:

• Conducted extensive experiments on hundreds of real-world networks.
• Evaluated 26 well-known link prediction algorithms.
• Analyzed the performance rankings produced by various evaluation metrics.

Main Results:

• Significant inconsistencies were found among different evaluation metrics, leading to divergent algorithm rankings.
• No single metric can comprehensively evaluate algorithm performance.
• Recommended using at least two metrics, such as AUC and Precision-Recall, or AUC and Normalized Discounted Cumulative Gain.

Conclusions:

• The choice of evaluation metrics critically affects link prediction algorithm assessment.
• A combination of metrics is necessary for credible performance evaluation.
• This work provides a foundation for developing standardized criteria for link prediction metric selection.