Regularly Truncated M-Estimators for Learning With Noisy Labels.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Sample selection is popular for learning with noisy labels in deep networks.
• Existing methods often incorrectly treat small-loss examples as clean and discard large-loss examples, overlooking potential information.
• This approach can be suboptimal due to the influence of noisy labels in selected samples and the underutilization of discarded data.

Purpose of the Study:

• To address the limitations of current sample selection methods in noisy label learning.
• To propose a novel approach that mitigates the negative impact of noisy labels in selected samples and leverages discarded data.
• To enhance the generalization capabilities of deep networks trained on datasets with label noise.

Main Methods:

• Introduction of regularly truncated M-estimators (RTME), a method that alternates between truncated and original M-estimator modes.
• Truncated M-estimators adaptively select small-loss examples, reducing the side-effects of noisy labels without prior noise rate knowledge.
• Original M-estimators incorporate large-loss examples, which may be clean or contain valuable information for generalization.

Main Results:

• Theoretical analysis demonstrates the label-noise-tolerant properties of the proposed strategies.
• Empirical results show RTME outperforms multiple baseline methods in learning with noisy labels.
• The method exhibits robustness across various noise types and levels.

Conclusions:

• RTME offers a simultaneous solution to the drawbacks of existing sample selection techniques in noisy label learning.
• The proposed approach effectively handles noisy labels, improving model generalization and robustness.
• RTME represents a significant advancement in training deep networks with imperfect data.