Differential Topic Models.

Area of Science:

• Computational Linguistics
• Statistical Modeling
• Data Mining

Background:

• Comparing document collections requires methods to identify shared and unique content.
• Existing topic models may not adequately capture inter-collection differences and similarities.
• Power-law distributions in topic-word frequencies are common and need specific modeling.

Purpose of the Study:

• To develop a differential topic model for comparative text mining.
• To model both similarities and differences between document collections.
• To incorporate prior knowledge, like vocabulary variations, into topic modeling.

Main Methods:

• Utilized hierarchical Bayesian nonparametric models, specifically the Pitman-Yor process.
• Introduced the transformed Pitman-Yor process (TPYP) to integrate prior knowledge.
• Developed an efficient sampling algorithm using data augmentation for the non-conjugate TPYP model.

Main Results:

• The model successfully discovers distinct aspects within different document collections.
• The proposed Markov Chain Monte Carlo (MCMC) algorithm significantly reduced test perplexity.
• Outperformed state-of-the-art models in document classification and ideology prediction tasks.

Conclusions:

• The differential topic model effectively captures comparative aspects of document collections.
• The TPYP and its associated sampling algorithm provide a robust framework for cross-collection modeling.
• This approach enhances the performance of downstream tasks like document classification.