Gray Learning From Non-IID Data With Out-of-Distribution Samples.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Training data integrity is crucial but often compromised, especially in non-independent and identically distributed (non-IID) datasets.
• Expert annotations can be unreliable, with out-of-distribution samples misclassified as in-distribution, leading to challenges in robust neural network training.

Purpose of the Study:

• To address the challenge of learning robust neural networks from datasets with unreliable labels and mixed data distributions.
• To leverage complementary labels, which indicate classes a sample does not belong to, to improve model generalization.

Main Methods:

• Introduced gray learning (GL), a novel approach utilizing both ground-truth and complementary labels.
• GL adaptively adjusts loss weights for different label types based on prediction confidence.
• Derived generalization error bounds grounded in statistical learning theory to demonstrate GL's effectiveness in non-IID settings.

Main Results:

• Gray learning (GL) achieves tight generalization error constraints even in non-IID settings.
• Experimental evaluations show GL significantly outperforms alternative robust statistics-based approaches.
• The method effectively handles datasets with a mixture of in- and out-of-distribution samples and unreliable labels.

Conclusions:

• Gray learning (GL) offers a robust solution for training neural networks with compromised data integrity.
• The adaptive weighting of ground-truth and complementary labels enhances model generalization in challenging non-IID scenarios.
• GL demonstrates superior performance compared to existing methods, paving the way for more reliable AI systems.