Bayesian inference for nonnegative matrix factorisation models.

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Analysis

Background:

• Nonnegative Matrix Factorization (NMF) is a common technique for dimensionality reduction and feature extraction.
• Standard NMF with Kullback-Leibler (KL) error often uses Expectation-Maximization (EM) for parameter estimation.
• Existing methods may lack flexibility for incorporating prior information or advanced inference.

Purpose of the Study:

• To develop a comprehensive Bayesian statistical framework for KL-NMF.
• To extend standard KL-NMF by incorporating hierarchical generative models with prior components.
• To enable more powerful and flexible NMF models through Bayesian inference.

Main Methods:

• Formulated NMF within a hierarchical generative model including observation and prior components.
• Developed full Bayesian inference using variational Bayes and Monte Carlo methods.
• Retained model conjugacy for efficient computation and attractive properties.

Main Results:

• The proposed Bayesian framework encompasses standard KL-NMF algorithms as special cases.
• The method allows for the development of more powerful NMF models.
• Demonstrated effectiveness in model order selection and image reconstruction tasks.

Conclusions:

• The Bayesian approach offers a flexible and powerful extension to KL-NMF.
• The framework maintains desirable features of standard NMF, such as monotonic convergence.
• This work provides a robust methodology for advanced NMF applications.