Hard Sample Mining: A New Paradigm of Efficient and Robust Model Training.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep learning (DL) models achieve breakthroughs in computer vision (CV) and natural language processing (NLP).
• Training deep neural networks faces challenges like inefficiency and data biases despite computational advances.
• Hard sample mining (HSM) emerges as a key technique to improve training efficiency and model robustness.

Purpose of the Study:

• To systematically survey and analyze hard sample mining (HSM) methodologies in deep learning.
• To establish unified definitions for hard samples using sample complexity quantification.
• To provide a taxonomy of HSM approaches and identify future research frontiers.

Main Methods:

• Defining hard samples through rigorous sample complexity quantification criteria.
• Proposing a systematic taxonomy of existing hard sample mining approaches.
• Conducting an in-depth technical analysis of various HSM strategies.

Main Results:

• Unified definitions for hard samples are established.
• A comprehensive taxonomy categorizes HSM methods.
• Key research frontiers and future directions are identified.

Conclusions:

• Hard sample mining is crucial for efficient and robust deep learning model training.
• This survey consolidates HSM foundations and offers a roadmap for future advancements.
• The research facilitates the development of more generalizable and reliable AI models.