A drop-out mechanism for active learning based on one-attribute heuristics.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Active Learning (AL) trains models with minimal labeled data by selecting informative samples.
• Human annotators in AL can introduce systematic bias by over-relying on specific attributes.
• This bias can reduce the efficiency and accuracy of AL systems.

Purpose of the Study:

• To develop a mathematically grounded method for quantifying mislabeling probability based on single attributes.
• To introduce a novel dropout mechanism to mitigate attribute selection bias during human annotation.
• To evaluate the effectiveness of this mechanism in improving active learning performance.

Main Methods:

• Developed a method to calculate the probability of mislabeling attributed to single-attribute reliance.
• Implemented a novel dropout mechanism integrated into the annotation process to guide attribute selection.
• Evaluated the dropout mechanism across various active learning algorithms and heuristic strategies on diverse tasks.

Main Results:

• The proposed dropout mechanism significantly improved active learning performance.
• A minimum of 70% enhancement in effectiveness was observed across experiments.
• The mechanism effectively reduced the impact of human annotator bias on data labeling.

Conclusions:

• The novel dropout mechanism is a viable strategy for reducing bias in active learning.
• This approach enhances the reliability and accuracy of AL systems.
• Findings offer valuable insights for developing more robust intelligent systems.