Non-negative matrix factorization algorithms generally improve topic model fits

Area of Science:

• Computational statistics
• Machine learning
• Natural language processing

Background:

• Topic models are essential for analyzing large text datasets.
• Maximum-likelihood estimation (MLE) is a standard approach for fitting topic models.
• Existing MLE methods can be computationally intensive for large datasets.

Purpose of the Study:

• To develop faster and more accurate topic modeling methods for large datasets.
• To formally connect topic modeling MLE with non-negative matrix factorization (NMF).
• To leverage advances in NMF optimization for topic model fitting.

Main Methods:

• Revisiting maximum-likelihood estimation in topic models.
• Establishing a formal connection between topic model MLE and non-negative matrix factorization (NMF).
• Applying recent NMF optimization techniques to fit topic models.

Main Results:

• Demonstrated that advances in NMF optimization can efficiently fit topic models.
• Achieved significantly better fits in less time compared to existing topic modeling algorithms.
• Showed that the Expectation-Maximization (EM) algorithm for topic models is equivalent to classic NMF multiplicative updates.

Conclusions:

• Novel methods based on NMF provide a highly efficient approach for topic modeling.
• The "fastTopics" R package implements these advanced NMF-based topic modeling techniques.
• This work bridges the gap between NMF and topic modeling, offering practical benefits for large-scale text analysis.