Transformer and Pre-Transformer Model-Based Sentiment Prediction with Various Embeddings: A Case Study on Amazon...

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Sentiment analysis is crucial for consumer insights but faces challenges with ambiguous language and varied data.
• Selecting optimal models and embeddings for sentiment classification remains difficult.

Purpose of the Study:

• To comprehensively evaluate sentiment classification models across traditional, deep learning, and transformer paradigms.
• To compare static (GloVe, FastText) and contextual (BERT, DistilBERT) embeddings.
• To introduce an entropy-aware framework for model selection.

Main Methods:

• Comparative analysis of machine learning, deep learning (LSTM), and transformer models.
• Evaluation of static and transformer embeddings on Amazon datasets.
• Use of accuracy, precision, recall, F1-score, and cross-entropy for performance measurement.
• Qualitative analysis of misclassified samples.

Main Results:

• FastText embeddings outperform GloVe, especially in recall, due to subword information.
• Transformer models, particularly DistilBERT, achieve the highest accuracy (92%) and lowest cross-entropy (0.25).
• Results validated on a second dataset, showing consistent trends.

Conclusions:

• Transformer models, especially DistilBERT, provide well-calibrated and accurate sentiment predictions.
• The proposed entropy-aware framework supports informed, context-sensitive model selection.
• Findings contribute to advancing sentiment analysis and sustainable AI practices.