Bayesian Estimation of Latently-grouped Parameters in Undirected Graphical Models

Area of Science:

• Machine Learning
• Statistical Modeling
• Network Analysis

Background:

• Large-scale undirected graphical models, common in social and biological networks, exhibit parameter redundancy.
• Efficient parameter learning is crucial for handling complex network structures.

Purpose of the Study:

• To develop a Bayesian method for inferring unknown parameter groups in graphical models.
• To introduce novel approximation algorithms for posterior inference in these models.

Main Methods:

• A Dirichlet process prior was imposed on model parameters.
• Two approximation algorithms were proposed: Metropolis-Hastings with auxiliary variables and Gibbs sampling with stripped Beta approximation (Gibbs_SBA).

Main Results:

• Both proposed algorithms outperformed conventional maximum likelihood estimation (MLE).
• Gibbs_SBA demonstrated performance comparable to exact likelihood Gibbs sampling.
• Models trained using Gibbs_SBA showed superior generalization on real-world Senate voting data compared to MLE.

Conclusions:

• The Bayesian approach effectively infers parameter groups, enhancing learning efficiency in graphical models.
• The proposed Gibbs_SBA algorithm offers a computationally efficient and accurate alternative to traditional methods.
• This method improves model generalization in complex network applications.