Radon-Sobolev Variational Auto-Encoders.

Area of Science:

• Machine Learning
• Probability Theory
• Computer Vision

Background:

• Generative models like Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs) rely on probability distances for quality assessment.
• Existing metrics such as Wasserstein, Jensen-Shannon, and Kullback-Leibler divergences have limitations, including lack of geodesic convexity and slow evaluation.
• These limitations hinder the optimization and performance of generative models.

Purpose of the Study:

• To introduce a new class of probability distances with built-in convexity properties.
• To address the shortcomings of existing probability distances in generative modeling.
• To develop an improved Variational Auto-Encoder utilizing these novel distances.

Main Methods:

• Introduced a novel class of probability distances with inherent convexity.
• Investigated the relationship between these new distances and existing paradigms like sliced (Radon) distances, reproducing kernel Hilbert spaces, and energy distances.
• Developed and implemented the Radon-Sobolev Variational Auto-Encoder (RS-VAE).

Main Results:

• The proposed distances exhibit desirable properties, including built-in convexity and fast implementations.
• The Radon-Sobolev Variational Auto-Encoder (RS-VAE) achieved high-quality results on standard generative datasets.
• Demonstrated the effectiveness of the new distances in enhancing generative model performance.

Conclusions:

• The developed convex probability distances offer a significant improvement over traditional metrics for generative models.
• The RS-VAE framework provides a powerful tool for generating high-quality data.
• This work contributes to advancing the field of generative modeling through improved distance metrics.