Stabilizing Training of Generative Adversarial Nets via Langevin Stein Variational Gradient Descent

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Generative Adversarial Networks (GANs) excel at learning data distributions but suffer from training instability, leading to mode collapse and performance degradation.
• Existing GAN training stabilization methods often compromise theoretical soundness and convergence.
• There is a need for robust methods to improve GAN training stability and performance.

Purpose of the Study:

• To propose a novel method, Langevin Stein Variational Gradient Descent (LSVGD), for stabilizing GAN training.
• To enhance the flexibility and efficiency of Stein Variational Gradient Descent (SVGD) for GANs.
• To improve particle diversity and generalization in GANs.

Main Methods:

• Introduced LSVGD, a particle-based variational inference method incorporating disturbances into update dynamics.
• Demonstrated that LSVGD simulates a Langevin process with the target distribution as its stationary distribution.
• Developed an efficient particle-based variational inference approach applicable to general GAN training procedures.

Main Results:

• LSVGD effectively stabilizes GAN training, mitigating issues like mode collapse and performance deterioration.
• The method enhances particle spread-out and diversity through implicit regularization.
• Experiments on synthetic and benchmark datasets (Cifar-10, Tiny-ImageNet, CelebA) show significant performance and stability improvements.

Conclusions:

• LSVGD offers a theoretically sound and practically efficient approach to stabilize GAN training.
• The method improves the quality and diversity of generated samples.
• LSVGD represents a significant advancement for reliable GAN model development.