Pseudo-document simulation for comparing LDA, GSDMM and GPM topic models on short and sparse text using Twitter data.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Data Science

Background:

• Topic models are essential for uncovering latent themes in documents.
• Short, sparse texts from social media pose challenges for traditional Latent Dirichlet Allocation (LDA).
• Existing coherence scores may be inadequate for evaluating topic models on sparse data.

Purpose of the Study:

• To compare the performance of Latent Dirichlet Allocation (LDA) with Gibbs Sampler Dirichlet Multinomial Model (GSDMM) and Gamma Poisson Mixture Model (GPM) for short, sparse texts.
• To introduce a novel simulation-based evaluation method for topic models.
• To assess topic model performance on Covid-19 related tweets.

Main Methods:

• Comparison of LDA, GSDMM, and GPM topic models.
• Development and application of a pseudo-document simulation for evaluation.
• Case study using Covid-19 pandemic tweets.

Main Results:

• Standard coherence scores demonstrated poor performance as an evaluation metric.
• The simulation-based approach indicated GSDMM and GPM may be superior to LDA for sparse data.
• GSDMM and GPM showed potential for improved topic generation on challenging datasets.

Conclusions:

• GSDMM and GPM are promising alternatives to LDA for analyzing short, sparse text data.
• The proposed simulation method offers a more reliable evaluation of topic models.
• Further research is warranted to validate these findings across diverse short-text datasets.