Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Traditional word vectors lack polysemy, hindering sentiment analysis.
• Recurrent Neural Networks face parallel training limitations and lower classification accuracy.
• Existing models struggle with nuanced text sentiment interpretation.

Purpose of the Study:

• To develop a novel sentiment classification model addressing limitations of traditional approaches.
• To enhance text sentiment analysis accuracy and model training efficiency.
• To leverage advanced deep learning techniques for improved sentiment detection.

Main Methods:

• Utilized Bidirectional Encoder Representations from Transformers (BERT) for word embeddings.
• Implemented a Sliced Bidirectional Gated Recurrent Unit (Sliced Bi-GRU) for feature extraction.
• Incorporated a Multi-head Self-Attention mechanism to capture word relationships sequentially.

Main Results:

• Achieved 74.37% classification accuracy on the Yelp 2015 dataset.
• Attained 62.57% classification accuracy on the Amazon dataset.
• Demonstrated superior training speed compared to most existing sentiment analysis models.

Conclusions:

• The proposed model effectively enhances text sentiment classification accuracy.
• The integration of BERT, Sliced Bi-GRU, and Self-Attention improves model performance.
• The model offers a promising solution for efficient and accurate sentiment analysis.