Automatic relevance determination in nonnegative matrix factorization with the β-divergence

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• Nonnegative Matrix Factorization (NMF) is crucial for dimensionality reduction.
• Estimating the optimal latent dimensionality (model order) is essential to prevent overfitting and maintain data fidelity.
• Existing methods for β-divergence-based NMF lack robust model order selection.

Purpose of the Study:

• To develop a Bayesian method for estimating latent dimensionality in NMF with β-divergence.
• To introduce a novel approach for automatic relevance determination (ARD) within the NMF framework.
• To propose efficient algorithms for maximum a posteriori (MAP) estimation.

Main Methods:

• A Bayesian model incorporating Automatic Relevance Determination (ARD) is proposed.
• Scale parameters in the priors of dictionary and activation matrices are tied.
• Majorization-Minimization (MM) algorithms are developed for MAP estimation.
• Inference drives a subset of scale parameters to a lower bound, pruning components.

Main Results:

• The proposed ARD-based Bayesian NMF effectively estimates latent dimensionality.
• The MM algorithms efficiently perform MAP estimation.
• Experimental results on synthetic and real-world datasets demonstrate robustness and efficacy.
• The method successfully prunes spurious components, improving model interpretability.

Conclusions:

• The developed Bayesian NMF with ARD offers a principled approach to model order selection.
• The proposed MM algorithms provide an efficient solution for parameter estimation.
• This method enhances the performance and reliability of NMF across various applications.