Outlier-trimmed dual-interval smoothing loss for sample selection in learning with noisy labels

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Real-world datasets frequently contain noisy labels, risking Deep Neural Network (DNN) overfitting due to high parameter complexity.
• Sample selection methods for Learning with Noisy Labels (LNL) identify clean and noisy samples by loss values, but temporal instability causes selection errors.
• Erroneous exclusion of clean samples and retention of noisy instances hinder the effectiveness of traditional sample selection in LNL.

Purpose of the Study:

• Propose a novel Outlier-Trimmed Dual-Interval Smoothing (OTDIS) loss function to enhance sample selection robustness and mitigate DNN overfitting to label noise.
• Address loss instability during iterative optimization by redefining noise levels through dual-interval estimation integrating temporal dynamics and sample distributions.
• Improve the reliability of sample selection by investigating the impact of outlier losses in early training stages.

Main Methods:

• Implement temporal smoothing using outlier-trimmed confidence interval lower bounds to enhance temporal robustness in sample selection.
• Utilize sample-space smoothing via clustering-based regrouping for distributionally stable loss estimates.
• Develop a dual-polarity training objective incorporating negative loss as a penalty and establish common and semi-supervised learning frameworks based on OTDIS loss.

Main Results:

• OTDIS loss significantly improves sample selection accuracy compared to existing methods.
• Achieved superior classification performance on benchmark datasets (MNIST, CIFAR) with synthetic noise.
• Demonstrated effectiveness on real-world noisy datasets including CIFAR-N, ANIMAL-10N, and WebVision.

Conclusions:

• OTDIS loss offers a robust solution for Learning with Noisy Labels (LNL) by stabilizing sample selection and reducing overfitting.
• The proposed method enhances DNN performance across various datasets, including those with real-world label noise.
• OTDIS provides a valuable tool for improving the reliability and accuracy of machine learning models trained on imperfect data.