Research on movie rating based on BERT-base model.

Area of Science:

• Natural Language Processing (NLP)
• Deep Learning
• Sentiment Analysis

Background:

• Movie reviews are vital for user film selection and content recommendation.
• Manual review classification is time-consuming, labor-intensive, and subjective.
• Deep learning models like BERT offer automated classification but have limitations in handling long texts and potential biases.

Purpose of the Study:

• To improve the BERT model's performance in movie review classification.
• To address challenges in capturing long-range dependencies and local features in lengthy reviews.
• To mitigate model bias arising from sensitive attributes like gender and race.

Main Methods:

• Implemented a dynamic positional offset encoding mechanism based on attention.
• Introduced a dynamic weighted fusion pooling strategy combining average, maximum, and self-attention pooling.
• Mitigated sensitive attributes (gender, race) during preprocessing and used data augmentation (EDA, noise injection) for neutral samples.

Main Results:

• The enhanced BERT model demonstrated improved feature extraction and positional information processing.
• Bias reduction techniques and data augmentation enhanced model generalization.
• Achieved a 0.73% increase in F1 score and a 0.90% improvement in accuracy on the IMDb dataset.

Conclusions:

• The proposed enhancements effectively improve BERT's capability for movie review classification.
• The study successfully addressed limitations related to long-range dependencies and data bias.
• The modified model offers a more accurate, fair, and robust solution for automated review analysis.