Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding.

Area of Science:

• Machine Learning
• Unsupervised Learning
• Data Clustering

Background:

• Clustering is a fundamental unsupervised learning task.
• Existing methods may struggle with complex data distributions.
• Generative models offer a powerful approach for density estimation and clustering.

Purpose of the Study:

• To develop a novel unsupervised generative clustering framework.
• To combine the strengths of variational information bottleneck and Gaussian mixture models.
• To provide an efficient and effective method for data clustering.

Main Methods:

• Developed a framework integrating variational information bottleneck (VIB) with Gaussian mixture models (GMM).
• Modeled the latent space using a mixture of Gaussians.
• Derived a generalized Evidence Lower Bound (ELBO) for the model's cost function.
• Implemented a variational inference algorithm using neural networks for coders' mappings.
• Optimized the model using stochastic gradient descent with Markov sampling.

Main Results:

• The proposed framework demonstrates efficient clustering performance on real datasets.
• The derived generalized ELBO provides a robust cost function.
• The variational inference algorithm effectively computes the bound.

Conclusions:

• The developed unsupervised generative clustering framework is efficient and effective.
• The integration of VIB and GMM offers a promising direction for clustering.
• The method shows potential for various data analysis applications.