Towards a Theoretical Understanding of Semi-Supervised Learning Under Class Distribution Mismatch

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Semi-supervised learning (SSL) struggles with class distribution mismatch, where unlabeled data contain categories absent in labeled data.
• Traditional SSL methods degrade due to unknown category instances invading the learning process.
• Theoretical analysis of SSL under class distribution mismatch remains an open research area.

Purpose of the Study:

• To theoretically analyze the excess risk in SSL under class distribution mismatch.
• To identify the core contributors to SSL error in mismatch scenarios.
• To propose a novel mechanism for improving SSL performance under distribution shifts.

Main Methods:

• Development of the Bi-Objective Optimization Mechanism (BOOM) for theoretical analysis.
• Analysis of excess risk, decomposing it into pseudo-labeling error and invasion error.
• Identification of two key optimization objectives: high-quality pseudo-labels and adaptive instance weighting.

Main Results:

• BOOM reveals SSL error, stemming from pseudo-labeling and invasion errors, as the primary cause of excess risk.
• The study identifies high-quality pseudo-labels and adaptive instance weights as crucial for mitigating these errors.
• A novel approach based on BOOM is proposed as an effective baseline for SSL under mismatch.

Conclusions:

• The Bi-Objective Optimization Mechanism (BOOM) provides a theoretical framework for understanding SSL under class distribution mismatch.
• Addressing pseudo-labeling and invasion errors through optimized objectives is key to robust SSL.
• The proposed method demonstrates effectiveness on benchmark and real-world datasets, offering a strong baseline.