Renormalization Analysis of Topic Models.

Area of Science:

• Computational Linguistics
• Statistical Physics
• Machine Learning

Background:

• Parameter tuning for big data machine learning models is computationally expensive, often relying on slow grid search.
• Statistical physics offers advanced optimization techniques applicable to machine learning challenges.

Purpose of the Study:

• To develop a faster method for determining the optimal number of topics in topic modeling.
• To apply statistical physics principles to accelerate machine learning parameter optimization.

Main Methods:

• Developed a renormalization procedure inspired by statistical physics.
• Combined renormalization with Renyi entropy for efficient topic number searching.
• Applied the method to probabilistic Latent Semantic Analysis (pLSA), Variational Expectation-Maximization for Latent Dirichlet Allocation (VLDA), and Granulated Gibbs Sampling for Latent Dirichlet Allocation (GLDA).

Main Results:

• The renormalization procedure demonstrated self-similar behavior in topic modeling outputs, enabling optimization.
• Experiments showed the method finds an approximation of the optimal number of topics at least 30 times faster than grid search.
• No significant loss of quality was observed compared to traditional methods.

Conclusions:

• The developed renormalization technique offers a substantial speedup for topic number optimization in machine learning.
• This approach leverages statistical physics to overcome computational bottlenecks in big data analysis.
• The method is effective across different topic modeling algorithms and datasets.