TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification.

Area of Science:

• Natural Language Processing
• Machine Learning
• Data Analytics

Background:

• Sentiment classification aims to analyze opinions from data, with recent advancements driven by BERT (Bidirectional Encoder Representations from Transformers) models.
• While BERT models offer strong performance, further fine-tuning with domain-specific data is crucial for enhanced accuracy in sentiment analysis tasks.

Purpose of the Study:

• To develop TopicBERT, a BERT model fine-tuned for recognizing topics at corpus, word, and sentence levels to improve sentiment classification.
• To introduce two TopicBERT variants: TopicBERT-ATP (aspect topic prediction) and TopicBERT-TA (topic augmentation).

Main Methods:

• TopicBERT-ATP utilizes an auxiliary task for topic information, with topics predetermined by Latent Dirichlet Allocation (LDA) and collapsed Gibbs sampling.
• TopicBERT-TA directly injects topic representation into a topic augmentation layer, allowing dynamic topic changes during training.
• Both variants were evaluated on SemEval 2014 Task 4 datasets.

Main Results:

• Both TopicBERT-ATP and TopicBERT-TA achieved state-of-the-art performance in sentiment classification across two domains.
• Direct topic augmentation (TopicBERT-TA) demonstrated superior performance compared to further training methods.
• Ablation, parameter, and complexity studies provided comprehensive analyses of the models' effectiveness.

Conclusions:

• TopicBERT represents a significant advancement in sentiment classification by integrating topic-level understanding.
• TopicBERT-TA offers a more effective approach than traditional fine-tuning for sentiment analysis tasks.
• The findings suggest that incorporating dynamic topic augmentation directly into the model architecture yields superior results.