How to handle noisy labels for robust learning from uncertainty.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) often encounter performance degradation when trained with large amounts of noisy labels due to the memorization effect and overfitting.
• Existing methods primarily rely on small loss tricks to address robust training challenges with noisy labels.
• The relationship between network uncertainty and label cleanliness is crucial for improving training stability.

Purpose of the Study:

• To analyze the relationship between uncertainties and clean labels in deep neural network training.
• To develop a novel robust training method that utilizes both small loss tricks and uncertainty-based label selection.
• To enhance the generalization performance of DNNs trained with extremely noisy labels.

Main Methods:

• Introduced Uncertain Aware Co-Training (UACT), a new training methodology.
• UACT incorporates a small loss trick alongside a mechanism to select potentially clean labels based on network uncertainty.
• The method leverages inherent network uncertainty to guide the learning process.

Main Results:

• UACT effectively avoids overfitting DNNs even with highly noisy labels.
• The proposed method achieves strong generalization performance by utilizing acquired network uncertainty.
• Experimental results on benchmark datasets (MNIST, CIFAR-10, CIFAR-100, T-ImageNet, News) demonstrate superior performance compared to state-of-the-art algorithms.

Conclusions:

• Uncertain Aware Co-Training (UACT) offers a robust solution for training deep neural networks with noisy labels.
• By effectively utilizing label uncertainty, UACT improves model generalization and mitigates overfitting.
• The proposed technique represents a significant advancement in handling noisy data in deep learning applications.