Distributionally Robust and Multi-Objective Nonnegative Matrix Factorization.

Area of Science:

• Machine Learning
• Data Analysis
• Signal Processing

Background:

• Nonnegative Matrix Factorization (NMF) is a dimensionality reduction technique for nonnegative data.
• The performance of NMF heavily relies on the assumed noise model, which is often unknown or difficult to estimate in real-world applications.
• This limitation hinders the reliability and robustness of standard NMF approaches.

Purpose of the Study:

• To address the challenge of unknown noise models in Nonnegative Matrix Factorization (NMF).
• To introduce a novel multi-objective NMF (MO-NMF) framework that integrates multiple objective functions.
• To develop a distributionally robust NMF (DR-NMF) solution that minimizes the maximum error across objectives, enhancing robustness to noise model uncertainty.

Main Methods:

• Formulated a multi-objective NMF (MO-NMF) problem by combining several objective functions.
• Utilized Lagrange duality to optimize weights for a weighted-sum objective function.
• Developed a multiplicative update algorithm for minimizing the weighted sum and a dual approach inspired by the Frank-Wolfe algorithm for DR-NMF.

Main Results:

• Demonstrated the effectiveness of the proposed MO-NMF and DR-NMF approaches on synthetic, document, and audio datasets.
• Showcased that DR-NMF solutions are robust even when the noise model of the NMF problem is unknown.
• Validated the ability of the dual approach to find distributionally robust solutions.

Conclusions:

• The proposed MO-NMF framework provides a flexible way to incorporate multiple NMF objectives.
• DR-NMF offers a robust alternative to standard NMF when noise characteristics are uncertain.
• The method shows significant promise for applications where data noise is a significant concern.