Nonparametric Spherical Topic Modeling with Word Embeddings.

Area of Science:

• Computational Linguistics
• Natural Language Processing
• Machine Learning

Background:

• Traditional topic models fail to incorporate semantic relationships between words.
• Existing models use inappropriate distributions (categorical, Gaussian) for leveraging word embedding correlations.
• Distributional word representations show semantic consistency via metrics like cosine similarity.

Purpose of the Study:

• To develop a novel topic model that effectively utilizes semantic regularities in language.
• To leverage directional word representations for improved topic modeling.
• To propose an efficient inference method for the new model.

Main Methods:

• Utilized the von Mises-Fisher distribution to model word density on a unit sphere, suitable for directional data.
• Employed a Hierarchical Dirichlet Process as the base topic model.
• Developed an efficient inference algorithm using Stochastic Variational Inference.

Main Results:

• The proposed model successfully exploits semantic structures from word embeddings.
• The model flexibly discovers an appropriate number of topics.
• Demonstrated superior performance in topic coherence compared to competitive methods on two text corpora.
• Achieved efficient inference.

Conclusions:

• The von Mises-Fisher distribution is effective for topic modeling with semantically rich word representations.
• The proposed Hierarchical Dirichlet Process-based model with Stochastic Variational Inference offers an efficient and effective approach to topic discovery.
• This method advances topic modeling by integrating semantic information and improving coherence.