Gaussian hierarchical latent Dirichlet allocation: Bringing polysemy back

Area of Science:

• Natural Language Processing
• Machine Learning
• Computational Linguistics

Background:

• Topic models like latent Dirichlet allocation (LDA) and Gaussian LDA (GLDA) uncover latent document representations.
• GLDA uses word embeddings but struggles with word polysemy, unlike LDA.
• Existing models lack the ability to simultaneously learn topic hierarchy and word polysemy.

Purpose of the Study:

• To introduce a novel topic model, Gaussian hierarchical latent Dirichlet allocation (GH-LDA), that addresses GLDA's limitations.
• To enhance the capture of word polysemy within topic modeling.
• To improve topic coherence and document representation accuracy.

Main Methods:

• Developed a hierarchical structure within Gaussian latent Dirichlet allocation.
• Introduced a model capable of representing documents using a hierarchy of topics.
• Employed extensive quantitative experiments on various corpora and word embeddings.

Main Results:

• GH-LDA significantly improves polysemy detection compared to Gaussian-based models.
• The proposed model offers more parsimonious topic representations than hierarchical LDA.
• Achieved superior topic coherence and held-out document predictive accuracy over GLDA and CGTM.

Conclusions:

• Gaussian hierarchical latent Dirichlet allocation effectively captures word polysemy and topic hierarchies simultaneously.
• The model provides a competitive alternative to existing methods like GLDA, with comparable time complexity.
• GH-LDA offers a more comprehensive approach to understanding document structure and word meaning.