A New Paradigm for Generative Adversarial Networks based on Randomized Decision Rules

Area of Science:

• Machine Learning
• Artificial Intelligence
• Statistics

Background:

• Generative Adversarial Networks (GANs) are powerful for training generative models but suffer from mode collapse, limiting generated data diversity.
• Mode collapse in GANs leads to a lack of variety in generated samples, hindering their application.
• Existing GAN training methods face challenges in achieving stable convergence and diverse outputs.

Purpose of the Study:

• To identify the root causes of mode collapse in GANs.
• To propose a novel GAN formulation addressing mode collapse using randomized decision rules.
• To develop a training method based on empirical Bayes principles for improved GAN performance.

Main Methods:

• Introduced a new GAN formulation with randomized decision rules, leading to discriminator convergence and generator convergence to a Nash equilibrium distribution.
• Proposed an empirical Bayes-like training method, treating the discriminator as a hyper-parameter.
• Utilized a stochastic gradient Markov chain Monte Carlo (MCMC) algorithm for simulating generators and stochastic gradient descent for discriminator updates.

Main Results:

• Established theoretical convergence of the proposed method to the Nash equilibrium.
• Demonstrated the method's effectiveness in addressing mode collapse and improving generated data diversity.
• Successfully applied the method to image generation, nonparametric clustering, and nonparametric conditional independence tests.

Conclusions:

• The proposed GAN formulation and training method effectively overcome mode collapse.
• The empirical Bayes approach with MCMC and SGD provides a stable and convergent training strategy.
• The method shows broad applicability beyond image generation, including statistical tasks.