Manifold adversarial training for supervised and semi-supervised learning.

Area of Science:

• Deep Learning
• Machine Learning
• Computer Vision

Background:

• Existing regularization and adversarial training methods in deep learning.
• Limitations of focusing on output space perturbations.
• Need for robust and compact data representations.

Purpose of the Study:

• To introduce Manifold Adversarial Training (MAT), a novel regularization technique for deep learning.
• To improve the robustness and compactness of data representations by considering latent space manifolds.
• To enhance model performance in supervised and semi-supervised learning tasks.

Main Methods:

• Developing an adversarial framework that targets the statistical manifold of latent representations.
• Utilizing a Gaussian Mixture Model (GMM) to derive latent feature spaces.
• Defining and promoting manifold smoothness against worst-case perturbations in the latent space.

Main Results:

• MAT achieves superior performance compared to state-of-the-art methods on benchmark datasets.
• Demonstrates significant improvements in both supervised and semi-supervised learning scenarios.
• MAT can be viewed as a generalization of the center loss method.

Conclusions:

• MAT offers a powerful approach for learning robust and compact data representations.
• The method effectively enhances deep learning model performance by leveraging latent space properties.
• Visualizations provide insights into adversarial examples and MAT's mechanism.