Progressive Stochastic Learning for Noisy Labels.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Large-scale machine learning relies on extensive labeled data, often sourced from crowdsourcing.
• Crowdsourced labels are frequently noisy, negatively impacting optimization algorithms like Stochastic Gradient Descent (SGD).
• Noisy labels particularly affect the primal variable updates in standard SGD.

Purpose of the Study:

• To introduce a robust SGD mechanism, Progressive Stochastic Learning (POSTAL), to mitigate the impact of noisy labels.
• To integrate curriculum learning (CL) principles into SGD for improved label processing.
• To enhance the reliability of primal variable updates in large-scale learning scenarios.

Main Methods:

• Developed POSTAL, a novel SGD mechanism combining curriculum learning with standard SGD.
• Introduced "screening losses" to order labels from reliable to noisy.
• Implemented a progressive learning approach, starting with reliable labels and incrementally incorporating noisy ones.

Main Results:

• POSTAL ensures robust primal variable updates, prioritizing reliable labels before noisy ones.
• Theoretical analysis derived the convergence rate of POSTAL with screening losses.
• Experimental results demonstrated POSTAL's superior effectiveness and robustness over existing methods on simulated and real-world crowdsourced data.

Conclusions:

• POSTAL offers a robust solution for large-scale learning problems with noisy crowdsourced labels.
• The integration of curriculum learning and screening losses effectively handles label noise.
• POSTAL significantly improves performance and robustness compared to conventional SGD and other baselines.