Limited-memory fast gradient descent method for graph regularized nonnegative matrix factorization.

Area of Science:

• Machine Learning
• Data Mining
• Optimization Algorithms

Background:

• Graph Regularized Nonnegative Matrix Factorization (GNMF) is crucial for preserving data structure.
• Traditional Multiplicative Update Rule (MUR) for GNMF suffers from slow convergence.
• Existing Multiple Fast Gradient Descent (MFGD) methods face high computational costs due to dense Hessian matrices.

Purpose of the Study:

• To propose an efficient Limited-memory Fast Gradient Descent (L-FGD) method for optimizing GNMF.
• To address the computational inefficiencies of MFGD in GNMF.
• To enhance the convergence speed and practical performance of GNMF algorithms.

Main Methods:

• Developed an L-FGD method utilizing the Limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm.
• L-FGD approximates the inverse Hessian-gradient product for optimal step-size determination.
• Evaluated L-FGD's clustering performance on face image (ORL, PIE) and text (Reuters, TDT2) datasets using KL-divergence.

Main Results:

• L-FGD demonstrates superior efficiency compared to MFGD and MUR on real-world datasets.
• Experimental results confirm the effectiveness of L-FGD in optimizing KL-divergence based GNMF.
• The proposed L-FGD method achieves competitive or improved clustering performance.

Conclusions:

• L-FGD offers a computationally efficient and effective approach for GNMF optimization.
• The method successfully overcomes the limitations of previous GNMF optimization techniques.
• L-FGD is a promising alternative for tasks requiring fast and accurate matrix factorization with graph regularization.