Does Negative Sampling Matter? a Review With Insights Into its Theory and Applications

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Negative sampling has become a critical research area with broad applications in machine learning, computer vision, natural language processing, data mining, and recommender systems.
• Existing research lacks a unified framework to understand the fundamental impact and diverse methodologies of negative sampling.

Purpose of the Study:

• To investigate the fundamental impact and necessity of negative sampling in various AI domains.
• To propose a general framework that encompasses all existing negative sampling methods.
• To categorize and analyze the evolution and application of negative sampling strategies.

Main Methods:

• Historical analysis of negative sampling evolution across five distinct trajectories.
• Categorization of negative sample selection strategies: global, local, mini-batch, hop, and memory-based.
• Systematic classification of current negative sampling methodologies into five types: static, hard, GAN-based, Auxiliary-based, and In-batch.

Main Results:

• A comprehensive review and categorization of negative sampling techniques and their selection strategies.
• Identification of five distinct evolutionary paths for negative sampling.
• Analysis of practical applications across diverse scientific and technological fields.

Conclusions:

• Negative sampling plays a significant role across multiple AI disciplines.
• The proposed general framework provides a unified perspective on negative sampling methods.
• Further research directions and open problems in negative sampling are identified.