OTAMatch: Optimal Transport Assignment with PseudoNCE for Semi-supervised Learning.

Area of Science:

• Machine Learning
• Computer Science

Background:

• Semi-supervised learning (SSL) often uses self-training with consistency regularization.
• Existing methods employ threshold heuristics to manage label noise, but this can discard valuable data.
• Confirmation bias and underutilization of discriminative information are key challenges.

Purpose of the Study:

• To introduce OTAMatch, a novel SSL framework addressing limitations of threshold-based pseudo-labeling.
• To mitigate confirmation bias by effectively utilizing high-confidence data.
• To improve the robustness and performance of SSL algorithms in noisy environments.

Main Methods:

• Reformulated pseudo-labeling as an optimal transport (OT) assignment problem, solved via convex minimization and the Sinkhorn-Knopp algorithm.
• Incorporated epsilon-greedy posterior regularization and curriculum bias correction for robust OT assignments.
• Introduced PseudoNCE to maximize mutual information and balance convergence speed with performance.

Main Results:

• OTAMatch demonstrated competitive performance across various SSL benchmarks.
• Achieved a significant 9.45% error rate reduction over SoftMatch on ImageNet (100K-label split).
• Showcased substantial outperformance against state-of-the-art SSL algorithms in challenging scenarios.

Conclusions:

• OTAMatch offers a robust and effective approach to semi-supervised learning, particularly in noisy conditions.
• The optimal transport formulation and integrated strategies enhance data utilization and model performance.
• The framework represents a significant advancement in addressing confirmation bias and improving SSL efficacy.