Least kth-Order and Rényi Generative Adversarial Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Information Theory

Background:

• Generative Adversarial Networks (GANs) are powerful generative models.
• Existing GAN loss functions can limit performance and training stability.
• Generalizing loss functions using information-theoretic measures offers potential improvements.

Purpose of the Study:

• To develop novel, generalized generator loss functions for GANs.
• To enhance the performance and stability of GANs through information-theoretic approaches.
• To explore the impact of parameterized loss functions on generative modeling.

Main Methods:

• Introduction of Least kth-order GAN (LkGAN) using kth-order absolute error.
• Proposal of Rényi-centric GAN loss functions based on Rényi cross-entropy.
• Equivalence established between minimizing LkGAN loss and kth-order Pearson-Vajda divergence.
• Demonstration of Rényi-centric loss reducing to original GAN loss as alpha approaches 1.
• Experimental validation on MNIST and CelebA datasets using DCGAN and StyleGAN architectures.

Main Results:

• LkGAN generalizes LSGANs, with k=2 recovering the LSGAN loss.
• Rényi-centric loss preserves the equilibrium point of original GANs.
• Proposed loss functions demonstrate performance benefits on MNIST and CelebA datasets.
• Improvements observed in generated image quality (Fréchet Inception Distance) and training stability.
• Extra degrees of freedom from parameters k and α lead to enhanced performance.

Conclusions:

• Parameterized information-theoretic measures effectively generalize GAN loss functions.
• The proposed LkGAN and Rényi-centric GANs offer improved performance and stability.
• The approach is applicable beyond GANs to other AI applications like fairness and privacy.
• This work provides a flexible framework for designing advanced generative models.