Block-Active ADMM to Minimize NMF with Bregman Divergences

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• Nonnegative matrix factorization (NMF) is crucial for dimensionality reduction and clustering analysis.
• Alternating Direction Method of Multipliers (ADMM) and its variants are popular for solving NMF problems.
• Existing NMF algorithms can be computationally intensive and require numerous auxiliary variables.

Purpose of the Study:

• To propose a novel block-active ADMM method for minimizing NMF problems with general Bregman divergences.
• To improve the efficiency and convergence speed of NMF algorithms.
• To reduce the number of auxiliary variables required in the optimization process.

Main Methods:

• A block-active ADMM approach is proposed for NMF.
• Subproblems within ADMM are solved using a block-coordinate-descent-type (BCD-type) method.
• Block selection is based on the stationary condition to optimize variable usage.

Main Results:

• The proposed block-active ADMM method uses significantly fewer auxiliary variables compared to existing algorithms.
• Numerical experiments demonstrate faster convergence rates for the new algorithm.
• The algorithm is theoretically proven to converge sublinearly to a stationary point.

Conclusions:

• The block-active ADMM method offers a more efficient and faster solution for NMF problems.
• This advancement benefits applications in machine learning, image processing, and computer vision.
• The proposed method shows superiority over previously developed NMF algorithms.