A drop-out mechanism for active learning based on one-attribute heuristics

Active Learning (AL) leverages the principle that machine learning models can achieve high accuracy with fewer labeled samples by strategically selecting the most informative data points for training. However, when human annotators provide these labels, their decisions might exhibit a systematic bias. For example, humans frequently rely on a limited subset of the available attributes, or even on a single attribute, when making decisions, as when employing fast and frugal heuristics. This paper introduces a mathematically grounded approach to quantify the probability of mislabeling based on one attribute. We present a novel dropout mechanism designed to influence the attribute selection process used in annotation, effectively reducing the impact of bias. The proposed mechanism is evaluated using multiple AL algorithms and heuristic strategies across diverse prediction tasks. Experimental results demonstrate that the dropout mechanism significantly enhances active learning (AL) performance, achieving a minimum 70% improvement in effectiveness. These findings highlight the mechanism's potential to improve the reliability and accuracy of AL systems, providing valuable insights for designing and implementing robust intelligent systems.