Delving Deep Into Label Smoothing.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Label smoothing is a key regularization technique for deep neural networks (DNNs).
• It aims to mitigate overfitting and boost classification accuracy by using soft labels.
• Current methods generate soft labels via a fixed average between uniform and hard labels.

Purpose of the Study:

• To develop a more effective method for generating reliable soft labels in DNNs.
• To introduce an Online Label Smoothing (OLS) strategy for improved model supervision.
• To enhance the robustness of DNNs against noisy labels.

Main Methods:

• The Online Label Smoothing (OLS) strategy generates soft labels dynamically.
• OLS utilizes statistics from the model's predictions for the target category.
• It constructs a more balanced probability distribution between target and non-target categories.

Main Results:

• OLS significantly improved classification performance on CIFAR-100, ImageNet, and fine-grained datasets.
• The proposed OLS method demonstrated superior robustness to noisy labels compared to existing techniques.
• Experiments confirmed enhanced classification accuracy using the same underlying classification models.

Conclusions:

• Online Label Smoothing (OLS) offers a more reliable and effective approach to regularization in DNNs.
• OLS enhances both classification accuracy and model robustness, particularly in the presence of noisy data.
• The method provides a promising direction for improving deep learning model training and generalization.