Combining active learning suggestions.

Area of Science:

• Machine Learning
• Data Science

Background:

• Active learning (AL) methods aim to reduce data annotation costs by selecting informative training instances.
• Numerous AL heuristics exist for classification tasks, but optimal selection varies by dataset.

Purpose of the Study:

• To investigate combining multiple active learning heuristics for improved training example selection.
• To determine if combined active learners outperform individual heuristics and passive learning.

Main Methods:

• Empirical comparison of active learning combination strategies on benchmark datasets.
• Integration of active learners with bandit algorithms and rank aggregation techniques.
• Evaluation of different reward functions for bandit-based active learning.

Main Results:

• Combined active learning approaches significantly outperform passive learning on large datasets.
• Combining active learners removes the necessity of a priori heuristic selection.
• Rank aggregation demonstrates robust performance in combining active learners.

Conclusions:

• Combining active learning methods offers a more effective and flexible approach to identifying informative training data.
• Bandit algorithms present challenges in reward definition, while rank aggregation provides a reliable combination strategy.