Searching to Exploit Memorization Effect in Deep Learning With Noisy Labels.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Science

Background:

• Robust learning from noisy labels is crucial for deep networks.
• Controlling sample selection to leverage the memorization effect remains challenging.
• Automated machine learning (AutoML) has shown success in related areas.

Purpose of the Study:

• To propose a novel bi-level optimization framework for controlling sample selection in robust learning.
• To enhance deep network performance by effectively utilizing the memorization effect.
• To develop an efficient method for finding optimal sample selection schedules.

Main Methods:

• A bi-level optimization strategy is employed, parameterizing the selection process in the upper-level and optimizing via model training in the lower-level.
• Semi-supervised learning algorithms are integrated to utilize noisy-labeled data as unlabeled data.
• The bi-level optimization problem is solved using Newton and cubic regularization methods, considering validation curvature.

Main Results:

• Both Newton and cubic regularization methods converge to approximate stationary points.
• Cubic regularization demonstrates superior performance in finding better local optima with reduced computation time.
• The proposed method achieves significant improvements over existing techniques on benchmark and real-world datasets.

Conclusions:

• The proposed bi-level optimization approach effectively controls sample selection for robust learning from noisy labels.
• The cubic regularization method offers an efficient and effective solution for the optimization problem.
• This research provides a more efficient alternative to existing AutoML approaches for optimizing sample selection schedules.