A supervised topic embedding model and its application

Area of Science:

• Natural Language Processing
• Machine Learning
• Data Science

Background:

• Topic modeling uncovers latent themes in documents using word co-occurrence.
• Word embedding maps words to semantic spaces based on local context.
• Topic embedding combines these by modeling topics within a word embedding space.

Purpose of the Study:

• To introduce rTopicVec, a supervised topic embedding model.
• To jointly model documents and their associated numerical labels using regression.
• To enable prediction of response variables for unlabeled documents.

Main Methods:

• Developed rTopicVec, a supervised topic embedding model incorporating regression.
• Proposed a regularized variant of rTopicVec.
• Evaluated models on predicting stock returns from news and movie ratings from reviews.

Main Results:

• rTopicVec achieved higher prediction accuracy than three baseline methods.
• The performance improvement was statistically significant.
• The model successfully predicted numerical labels for unlabeled documents.

Conclusions:

• rTopicVec offers an effective approach for supervised topic embedding.
• The model demonstrates strong performance in regression tasks involving text data.
• This method advances the prediction of document-associated variables.