Learning with noisy labels via clean aware sharpness aware minimization.

Area of Science:

• Machine Learning
• Computer Science
• Artificial Intelligence

Background:

• Leveraging large, imprecise datasets is crucial in machine learning.
• Sharpness Aware Minimization (SAM) enhances generalization with noisy labels via adversarial perturbations.
• SAM's effectiveness is limited by challenges in identifying correct perturbations with noisy data.

Purpose of the Study:

• To address the generalization bottleneck in SAM caused by noisy labels.
• To develop a novel algorithm that improves parameter perturbation strategies in the presence of label noise.
• To enhance the robustness and performance of models trained on datasets with imperfect labels.

Main Methods:

• Theoretical analysis of parameter perturbation direction mismatch between clean and noisy samples.
• Development of Clean Aware Sharpness Aware Minimization (CA-SAM) algorithm.
• Dynamic data division into likely clean and noisy subsets based on model history.
• Utilizing likely clean samples to guide parameter perturbation direction.
• Searching for flat minima in the loss landscape to align noisy samples while preserving clean ones.

Main Results:

• CA-SAM effectively identifies and utilizes likely clean samples to guide adversarial perturbations.
• The algorithm successfully restricts gradient perturbation for noisy samples, aligning them with clean samples.
• Comprehensive experiments on diverse benchmark datasets demonstrate CA-SAM's superior performance.
• CA-SAM significantly outperforms existing state-of-the-art approaches in noisy label learning.

Conclusions:

• CA-SAM offers a robust solution to the challenges of noisy label learning.
• The proposed method enhances model generalization by effectively handling label noise.
• CA-SAM represents a significant advancement in developing reliable machine learning models from imperfect data.