A semi supervised approach to Arabic aspect category detection using Bert and teacher-student model

Area of Science:

• Natural Language Processing
• Machine Learning
• Sentiment Analysis

Background:

• Aspect-based sentiment analysis research is extensive in English but limited for Arabic, particularly in aspect category detection.
• Existing methods often rely on supervised learning and large labeled datasets, posing challenges for resource-scarce languages like Arabic.

Purpose of the Study:

• To implement a semi-supervised self-training, noisy student framework to enhance aspect category detection for Arabic.
• To improve the performance of the AraBERT v02 deep learning model using this framework on hotel review datasets.

Main Methods:

• A four-step framework involving a teacher model trained on labeled data, pseudo-label generation for unlabeled data, a noisy student model trained on combined data, and model ensembling.
• Utilized the SemEval 2016 hotel review dataset and the Hotel Arabic-Reviews Dataset (HARD) 2016, processing approximately 1 million sentences.

Main Results:

• The ensembled teacher-student model showed a 0.3% micro F1 improvement over the initial noisy student implementation for aspect category prediction.
• Achieved a 1% increase in micro F1 compared to the teacher model, outperforming baselines and other deep learning models.

Conclusions:

• The proposed semi-supervised noisy student framework effectively enhances Arabic aspect category detection capabilities.
• The ensembled AraBERT model demonstrates superior performance, addressing the research gap in Arabic aspect-based sentiment analysis.