Learning Harmonium models with infinite latent features.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Statistical Modeling

Background:

• Undirected latent variable models are crucial for various tasks but face challenges in determining the optimal number of hidden units.
• Bayesian nonparametrics have addressed model selection in directed models, but their application to undirected latent variable models is limited.

Purpose of the Study:

• To introduce the infinite exponential family Harmonium (iEFH), a bipartite undirected latent variable model.
• To address the challenge of automatically determining the number of latent units in undirected models.
• To present extensions for heterogeneous data (multiview iEFH) and supervised learning (infinite maximum-margin Harmonium - iMMH).

Main Methods:

• Development of the infinite exponential family Harmonium (iEFH) model.
• Introduction of multiview iEFH for heterogeneous data and infinite maximum-margin Harmonium (iMMH) for supervised learning.
• Application of variational inference algorithms for model parameter learning.

Main Results:

• The proposed iEFH model automatically determines the number of latent units from an unbounded pool, avoiding manual selection.
• Experiments on real image and text datasets demonstrate the effectiveness of iEFH and iMMH.
• The methods show computational competitiveness due to the automated selection of latent units.

Conclusions:

• The iEFH and iMMH models successfully leverage Bayesian nonparametrics and max-margin learning for undirected latent variable models.
• These methods expand the applicability of Bayesian nonparametrics to learning the structures of undirected latent variable models.
• The approach offers a computationally efficient and effective solution for learning latent variable models without pre-specifying the number of latent units.