Variational inference with Gaussian mixture model and householder flow.

Area of Science:

• Machine Learning
• Deep Generative Models
• Computer Vision

Background:

• Variational auto-encoders (VAEs) are powerful deep generative models, but their performance is limited by the flexibility of approximate posterior distributions.
• Standard VAEs often assume simple distributions (e.g., diagonal Gaussian) which cannot capture complex true posterior distributions.
• This limitation impacts the tractability and overall effectiveness of VAEs in modeling intricate data.

Purpose of the Study:

• To propose a novel method for designing flexible and arbitrarily complex approximate posterior distributions in VAEs.
• To enhance the representational capacity of VAEs by overcoming the limitations of standard posterior approximations.
• To achieve state-of-the-art results on benchmark datasets and challenging medical image data.

Main Methods:

• Constructing an initial density using a Gaussian mixture model with diagonal covariance matrices.
• Applying a sequence of invertible Householder transformations to iteratively increase the complexity of the approximate posterior distribution.
• Redefining the variational lower bound using its upper bound due to the intractability of KL-divergence between mixture densities.

Main Results:

• The proposed method achieves new state-of-the-art results on benchmark datasets like MNIST, Fashion-MNIST, and Omniglot.
• Demonstrates significant improvements in flexibility of posterior distribution modeling on challenging Histopathology medical image data.
• The Householder transformation approach effectively enhances the VAE's ability to approximate complex posterior distributions.

Conclusions:

• The novel VAE approach with Householder transformations offers a more flexible and powerful alternative to standard VAEs.
• This method effectively addresses the limitations of fixed, simple posterior distributions in deep generative modeling.
• The enhanced flexibility leads to superior performance, particularly on complex datasets such as medical images.