Gradient-aware learning for joint biases: Label noise and class imbalance

Area of Science:

• Machine Learning
• Computer Science
• Artificial Intelligence

Background:

• Large-scale datasets often contain data biases like class imbalance and label noise, posing significant challenges for deep learning models.
• Existing methods for mitigating these biases, such as re-weighting or regularization, often focus on class imbalance and may fail when both biases are present simultaneously, leading to overfitting of noisy labels.

Purpose of the Study:

• To propose a novel gradient-aware learning method designed to effectively handle the combined challenges of class imbalance and label noise in deep learning.
• To implicitly decouple the effects of label noise and class imbalance within the deep network architecture.

Main Methods:

• A gradient-aware learning strategy is employed, involving selective updates of crucial parameters and rectification of redundant parameter updates.
• This update rule is applied to both the encoder and classifier components of the deep network to address the dual biases.
• The method aims to implicitly disentangle the influences of label noise and class imbalance during the training process.

Main Results:

• Experimental results demonstrate the effectiveness of the proposed gradient-aware learning method on both synthetic and real-world datasets exhibiting combined biases.
• The method successfully mitigates the negative impact of class imbalance and label noise, leading to improved model performance.

Conclusions:

• The proposed gradient-aware learning method offers a robust solution for deep learning models trained on datasets with simultaneous class imbalance and label noise.
• This approach provides a way to implicitly decouple these biases, enhancing model generalization and performance in practical applications.