Shattering Distribution for Active Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Active learning (AL) aims to minimize labeling effort by strategically selecting data points for annotation.
• Existing AL methods often rely on hypothesis pruning, which can be sensitive to initial assumptions and updates.
• The performance of hypothesis-pruning strategies can be inconsistent, especially with adversarial examples or noisy labels.

Purpose of the Study:

• To propose a novel active learning strategy based on distribution shattering, independent of hypothesis estimation.
• To theoretically and empirically demonstrate the benefits of distribution shattering in reducing label complexity and error disagreement.
• To introduce the Shattered Distribution-based AL (SDAL) algorithm for efficient sample selection.

Main Methods:

• Developed a distribution-shattering strategy by halving the number density of the input distribution.
• Derived the Shattered Distribution-based AL (SDAL) algorithm to split the shattered distribution into representative samples.
• Conducted empirical evaluations on benchmark datasets, including experiments with adversarial examples and noisy labels.

Main Results:

• Sampling from a shattered distribution demonstrably reduces label complexity and error disagreement.
• The SDAL algorithm effectively utilizes the halving and querying abilities for real-world AL tasks with limited labels.
• Experiments confirmed the theoretical insights regarding the performance differences between hypothesis-pruning and distribution-shattering strategies.

Conclusions:

• The distribution-shattering approach offers a robust alternative to hypothesis-pruning in active learning.
• SDAL provides an effective method for selecting informative samples, enhancing learning efficiency with reduced labeling costs.
• The proposed strategy shows promise for improving active learning performance in challenging scenarios like noisy data.