Max-Margin Deep Generative Models for (Semi-)Supervised Learning.

Area of Science:

• Machine Learning
• Deep Learning
• Generative Models

Background:

• Deep generative models (DGMs) excel at capturing complex data distributions through multilayered representations and inference.
• However, DGMs often exhibit insufficient discriminative ability for predictive tasks.
• Existing methods struggle to balance generative power with strong classification performance.

Purpose of the Study:

• To introduce max-margin deep generative models (mmDGMs) and their class-conditional variant (mmDCGMs).
• To enhance the predictive performance of DGMs by integrating the max-margin learning principle.
• To maintain the generative capabilities of DGMs while improving their discriminative power in supervised and semi-supervised settings.

Main Methods:

• Proposed max-margin deep generative models (mmDGMs) and class-conditional variants (mmDCGMs).
• In semi-supervised learning, utilized max-margin classifier predictions for efficient pseudo-labeling instead of full posterior inference.
• Introduced max-margin and label-balance regularization terms for unlabeled data.
• Developed an efficient doubly stochastic subgradient algorithm for optimization.

Main Results:

• Max-margin learning significantly boosts the prediction performance of DGMs while preserving generative ability.
• In supervised learning, mmDGMs using convolutional neural networks are competitive with leading fully discriminative networks.
• In semi-supervised learning, mmDCGMs achieve state-of-the-art classification results on multiple benchmarks with efficient inference.

Conclusions:

• Max-margin learning is a powerful principle for enhancing DGMs' discriminative capabilities.
• mmDGMs and mmDCGMs offer a robust framework for improved supervised and semi-supervised learning.
• The proposed models effectively balance generative and discriminative objectives, advancing the field of deep learning.