Learning Generative Models Using Denoising Density Estimators.

Area of Science:

• Machine Learning
• Unsupervised Learning
• Probabilistic Modeling

Background:

• Estimating sample density and generating new samples are core challenges in unsupervised machine learning.
• Existing generative models often rely on specific network architectures or complex solvers.

Purpose of the Study:

• Introduce a novel generative model based on denoising density estimators (DDEs).
• Develop a technique to directly minimize Kullback-Leibler (KL) divergence for generative modeling.
• Offer an alternative to normalizing flows and continuous normalizing flows.

Main Methods:

• Utilized scalar functions parametrized by neural networks as denoising density estimators (DDEs).
• Developed an algorithm to directly minimize the Kullback-Leibler (KL) divergence.
• Proved the convergence guarantees of the proposed algorithm.

Main Results:

• Demonstrated substantial improvements in density estimation accuracy.
• Achieved competitive performance in training generative models.
• Showcased a method that does not require specific network architectures or ODE solvers.

Conclusions:

• The proposed DDE-based generative model offers an effective approach for density estimation and sample generation.
• The direct KL-divergence minimization technique is theoretically sound and practically effective.
• This method provides a flexible and efficient alternative to existing generative modeling techniques.