Deep Mixture Generative Autoencoders.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Variational Autoencoders (VAEs) are prominent unsupervised generative models for learning data representations.
• Classical Gaussian mixture models offer a framework for data clustering and density estimation.

Purpose of the Study:

• To extend deep variational frameworks by proposing a Mixture of Variational Autoencoders (MVAE).
• To enforce separation between latent spaces of different encoders using the d-variable Hilbert-Schmidt independence criterion (dHSIC).
• To develop a mechanism for determining the optimal number of VAE components for a given task.

Main Methods:

• Implementing each MVAE component with a variational encoder and a subdecoder.
• Utilizing the dHSIC to ensure distinct latent space representations for each component.
• Employing the differentiable categorical Gumbel-softmax distribution for end-to-end trainable dropout masking.

Main Results:

• The MVAE model successfully learns rich latent data representations.
• The model demonstrates the ability to uncover additional underlying data representation factors.
• Experiments validate the effectiveness of the proposed MVAE architecture.

Conclusions:

• The proposed MVAE framework effectively combines Gaussian mixture concepts with deep variational learning.
• MVAE enables the discovery of richer and more diverse latent data representations.
• The method provides a robust approach for unsupervised learning and feature discovery.