Generative Adversarial Training for Supervised and Semi-supervised Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Neural networks are crucial in research, but adversarial training (AT) has limitations in generating worst-case perturbations.
• Existing AT methods often rely on approximate solutions and specific smoothness functions, limiting performance improvements.

Purpose of the Study:

• To introduce Generative Adversarial Training (GAT), a novel methodology for supervised and semi-supervised learning.
• To address the limitations of current AT methods by proposing a minimax game formulation for perturbation generation and classifier training.

Main Methods:

• Formulated the learning task as a minimax game: a perturbation generator maximizes output deviation, while a classifier minimizes prediction error.
• Developed a new adversarial loss function based on cross-entropy to solve the minimax optimization problem.
• Implemented a trajectory-preserving alternating update strategy for stable GAT training.

Main Results:

• GAT significantly improves model smoothness and confidence compared to standard AT.
• Experiments show GAT outperforms state-of-the-art AT methods on benchmark datasets for supervised and semi-supervised learning.
• GAT demonstrates particular effectiveness in semi-supervised learning with limited labeled data.

Conclusions:

• Generative Adversarial Training (GAT) offers a superior approach to enhancing neural network generalization.
• The proposed minimax game and stable training strategy effectively improve model robustness and performance.
• GAT shows strong potential for applications in supervised and semi-supervised learning, especially in low-data regimes.