Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors.

Area of Science:

• Machine Learning
• Computational Biology
• Statistical Inference

Background:

• Latent variable models are crucial for complex data analysis.
• Variational inference methods approximate intractable posterior distributions.
• Existing approximate posteriors like normalizing flows and adversarial networks have limitations.

Purpose of the Study:

• To introduce Denoising Diffusion Variational Inference (DDVI), a novel black-box variational inference algorithm.
• To leverage diffusion models as flexible and powerful approximate posteriors.
• To improve inference and learning in deep latent variable models.

Main Methods:

• DDVI utilizes diffusion models for iterative refinement in latent space.
• A novel regularized evidence lower bound (ELBO) inspired by the wake-sleep algorithm is employed for training.
• The method is compatible with black-box variational inference frameworks.

Main Results:

• DDVI outperforms alternative approximate posterior methods, including normalizing flows and adversarial networks.
• The algorithm demonstrates improved inference and learning on common benchmarks.
• DDVI shows superior performance in inferring latent ancestry from human genomes on the 1000 Genomes dataset.

Conclusions:

• DDVI offers an effective and easy-to-implement approach for variational inference.
• Diffusion-based variational posteriors provide a more expressive and accurate approximation.
• DDVI holds significant promise for applications in machine learning and computational biology.