Probabilistic Topic Modeling With Transformer Representations.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Artificial Intelligence

Background:

• Topic modeling traditionally relied on Bayesian graphical models.
• Recent advances leverage transformer embeddings for topic discovery.
• Existing transformer-based methods often use simple clustering, lacking probabilistic depth.

Purpose of the Study:

• To propose the Transformer-Representation Neural Topic Model (TNTM).
• To integrate transformer-based embeddings with probabilistic topic modeling.
• To enhance topic coherence and diversity in NLP.

Main Methods:

• Developed TNTM, unifying transformer embeddings with probabilistic modeling.
• Utilized the Variational Autoencoder (VAE) framework for efficient inference.
• Leveraged transformer-based embedding spaces for topic representation.

Main Results:

• TNTM achieves state-of-the-art performance in embedding coherence.
• The model maintains high topic diversity.
• Experimental results demonstrate competitive performance against existing methods.

Conclusions:

• TNTM offers a powerful hybrid approach to topic modeling.
• The model successfully combines the strengths of transformer embeddings and probabilistic methods.
• TNTM provides a flexible and effective tool for uncovering latent topics in text data.