Feature Noise Boosts DNN Generalization Under Label Noise.

Area of Science:

• Machine Learning
• Deep Learning
• Artificial Intelligence

Background:

• Label noise in training data significantly degrades the generalization capabilities of deep neural networks (DNNs).
• Existing methods struggle to effectively mitigate the negative impacts of label noise on DNN performance.

Purpose of the Study:

• To introduce and theoretically validate a novel Feature Noise (FN) method for enhancing DNN generalization under label noise.
• To provide a theoretical understanding of how FN improves generalization by constraining DNNs.

Main Methods:

• Directly injecting noise into the features of training data.
• Theoretical analysis to demonstrate how FN impacts generalization bounds and mutual information between weights and features.
• Qualitative analysis to identify optimal FN strategies for label noise scenarios.

Main Results:

• Theoretical analysis confirmed that label noise weakens DNN generalization by loosening generalization bounds.
• Feature Noise (FN) was shown to improve DNN generalization by imposing an upper bound on mutual information, thereby constraining the generalization bound.
• Extensive experiments demonstrated that FN significantly enhances the label noise generalization of state-of-the-art methods across popular datasets.

Conclusions:

• The proposed Feature Noise (FN) method offers a simple yet effective approach to improve deep neural network generalization in the presence of label noise.
• FN provides a theoretically grounded mechanism for enhancing model robustness against noisy labels.