Tackling Noisy Labels With Network Parameter Additive Decomposition.

Area of Science:

• Machine Learning
• Computer Science

Background:

• Over-parameterized deep networks struggle with noisy labels, leading to poor generalization.
• The memorization effect allows networks to learn clean data before noisy data, but early stopping fails to distinguish between them.

Purpose of the Study:

• To decouple the memorization of clean and mislabeled data in deep networks.
• To reduce the negative impact of mislabeled data on model generalization.

Main Methods:

• Additive decomposition of network parameters into two groups: σ (clean data memorization) and γ (mislabeled data memorization).
• Modulating the updates of σ and γ based on the memorization effect to prioritize clean data learning.
• Utilizing only σ parameters during testing to enhance generalization.

Main Results:

• The proposed method effectively separates the learning of clean and noisy data.
• Demonstrated superior performance compared to existing methods on simulated and real-world benchmarks.
• Significantly improved generalization ability in the presence of noisy labels.

Conclusions:

• Additive parameter decomposition is a viable strategy to combat noisy labels in deep learning.
• The method leverages the memorization effect for improved model robustness and performance.
• The findings offer a new direction for developing more resilient deep learning models.