Bayesian model averaging of naive Bayes for clustering

Area of Science:

• Machine Learning
• Artificial Intelligence
• Statistical Modeling

Background:

• Unsupervised learning aims to discover patterns in data without predefined labels.
• Naive Bayes classifiers are probabilistic models widely used for classification tasks.
• Model-averaging (MA) can improve the robustness and accuracy of statistical models.

Purpose of the Study:

• To propose a novel Bayesian model-averaging (MA) approach for learning unsupervised naive Bayes classification models.
• To introduce the expectation model-averaging (EMA) algorithm for efficient computation of the approximate MA model.
• To demonstrate the utility of the EMA algorithm for unsupervised feature subset selection.

Main Methods:

• Development of the expectation model-averaging (EMA) algorithm.
• Application of EMA for approximating a model-average over selective naive Bayes structures.
• Analysis of the EMA algorithm's time complexity compared to the expectation-maximization (EM) algorithm.

Main Results:

• The EMA algorithm yields a unique naive Bayes model approximating an MA over selective structures.
• The EMA algorithm achieves this with the same time complexity as the EM algorithm for maximum-likelihood estimation.
• The resulting model inherently provides information on the dependency of predictive variables on the cluster variable.

Conclusions:

• The proposed EMA algorithm offers an efficient method for unsupervised naive Bayes model learning.
• EMA facilitates approximate Bayesian model-averaging in clustering tasks.
• The EMA algorithm serves as an effective approach for unsupervised feature subset selection by revealing variable-cluster dependencies.