Max-Margin Deep Diverse Latent Dirichlet Allocation With Continual Learning.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Deep probabilistic aspect models are crucial for document analysis and topic extraction.
• Existing models struggle with common, low-meaning words and integrating supervision into hierarchical topic models.

Purpose of the Study:

• To propose deep diverse latent Dirichlet allocation (DDLDA) for more meaningful semantic topics.
• To develop max-margin DDLDA (mmDDLDA) for discriminative topical representations using supervision.
• To introduce a continual learning approach for practical, real-world applications.

Main Methods:

• Introduction of shared topics in DDLDA to reduce common words.
• Development of a variational inference network for DDLDA generalization to mmDDLDA.
• Implementation of a continual hybrid method combining stochastic-gradient MCMC and variational inference.

Main Results:

• DDLDA and mmDDLDA yield more meaningful and discriminative topic representations.
• mmDDLDA achieves higher classification accuracy compared to DDLDA and existing models.
• The continual learning approach demonstrates effectiveness in preventing catastrophic forgetting and adapting to new tasks.

Conclusions:

• DDLDA and mmDDLDA offer superior performance in unsupervised and supervised topic modeling.
• The proposed models and continual learning method enhance the practicality and adaptability of deep topic models.