Neighborhood-Regularized Self-Training for Learning with Few Labels.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Deep neural networks (DNNs) training often requires extensive labeled data, posing a significant annotation challenge.
• Self-training is a semi-supervised learning technique that leverages unlabeled data but is susceptible to performance degradation due to noisy pseudo-labels.
• Addressing label noise is crucial for effective and efficient model training with limited supervision.

Purpose of the Study:

• To develop a robust self-training method that mitigates the impact of noisy pseudo-labels.
• To enhance the performance and reliability of deep neural networks in semi-supervised learning scenarios.
• To improve the efficiency of the training process by optimizing sample selection.

Main Methods:

• A neighborhood-based sample selection strategy is proposed to identify and filter out noisy pseudo-labels.
• The self-training process is stabilized by aggregating predictions across multiple training rounds.
• The approach focuses on the principle that similar labels often correspond to similar data representations.

Main Results:

• The proposed method demonstrates superior performance compared to existing self-training baselines, achieving an average performance gain of 1.83% on text and 2.51% on graph datasets.
• The data selection strategy effectively reduces pseudo-label noise by 36.8%.
• Significant time savings of 57.3% were observed compared to the best baseline method.

Conclusions:

• The neighborhood-based sample selection approach offers a promising solution for noisy pseudo-labels in self-training.
• The method enhances both the accuracy and efficiency of deep neural network training under limited supervision.
• This work contributes to advancing semi-supervised learning techniques for practical applications.