CrowdTeacher: Robust Co-teaching with Noisy Answers and Sample-Specific Perturbations for Tabular Data

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Ground truth labels are often unavailable in real-world applications.
• Existing models struggle with sparse, unreliable, or differing crowdsourced annotations.
• Co-teaching methods show promise for noisy labels but require adaptation for crowdsourcing.

Purpose of the Study:

• To develop a robust machine learning model for handling noisy crowdsourced labels.
• To improve classifier performance when dealing with sparse and unreliable annotations.
• To extend co-teaching principles for tabular data with crowdsourced labels.

Main Methods:

• CrowdTeacher perturbs samples in the input space based on annotation certainty.
• A co-teaching algorithm is adapted to accommodate perturbed samples and smaller tabular datasets.
• The model leverages aggregated annotations to guide sample perturbation and improve robustness.

Main Results:

• CrowdTeacher significantly boosts predictive power on both synthetic and real datasets.
• The approach demonstrates superior performance across various label density settings.
• Experiments show CrowdTeacher outperforms baseline methods, including individual annotation modeling and truth inference.

Conclusions:

• CrowdTeacher offers an effective solution for leveraging noisy crowdsourced data in machine learning.
• The proposed perturbation-based co-teaching method enhances classifier robustness and accuracy.
• This work advances the field of learning with imperfect labels, particularly for tabular data.