Nonnegative matrix factorization with Wasserstein metric-based regularization for enhanced text embedding.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Information Retrieval

Background:

• Nonnegative Matrix Factorization (NMF) is vital for text embedding.
• Standard NMF, based on bag-of-words, ignores contextual and semantic information.
• This limitation leads to a significant loss of semantic meaning in text representations.

Purpose of the Study:

• To propose a novel Nonnegative Matrix Factorization (NMF) scheme enhanced with Wasserstein metric regularization.
• To leverage semantic information by approximating a word-context matrix using the Wasserstein metric.
• To adapt existing NMF algorithms for the new Wasserstein-regularized NMF (NMF-WR) framework.

Main Methods:

• Developed a new NMF scheme incorporating a regularization term based on the Wasserstein metric.
• Utilized the symmetric and positive definite (SPD) properties of the word-context matrix within a manifold structure.
• Modified gradient computations to adapt three classes of numerical algorithms for NMF-WR.

Main Results:

• The proposed NMF-WR model demonstrated superior performance in text embedding tasks.
• Experiments on topic modeling and document clustering showed significant improvements over conventional NMF models.
• The NMF-WR framework effectively enhanced semantic representation in text data.

Conclusions:

• The novel NMF-WR framework significantly improves text embedding quality by incorporating semantic information.
• The proposed algorithms are effective for solving the Wasserstein-regularized NMF problem.
• NMF-WR offers a more reliable and interpretable approach to text representation for NLP tasks.