Culturally aware comparative evaluation of machine learning and deep learning models for cyberbullying detection
Related Experiment Videos
Golam Md Mohiuddin1, Md Shohel Sayeed2, Ong Lee Yeng3
1Faculty of Information Science and Technology, Multimedia University, Melaka, Malaysia.
Introduction:
The rapid growth of social media has increased the prevalence of cyberbullying, raising serious psychological and social concerns. As abusive language varies across demographic and cultural groups, automated detection systems must be evaluated beyond overall accuracy to assess their effectiveness in identifying culturally sensitive forms of cyberbullying. This study proposes a culturally aware evaluation framework to compare traditional machine learning and deep learning models across four demographic-related categories: age, gender, religion, and ethnicity.
Methods:
Experiments were conducted using the IEEE DataPort Fine-Grained Balanced Cyberbullying Dataset. Traditional machine learning models were developed using Term Frequency-Inverse Document Frequency (TF-IDF) feature representations, while deep learning architectures included TextCNN, Bidirectional Long Short-Term Memory (BiLSTM), Bidirectional Gated Recurrent Unit (BiGRU), and CNN-BiLSTM. Model performance was assessed using accuracy, precision, recall, F1-score, confusion matrices, and class wise cultural sensitivity analysis.
Results:
The Extra Trees Classifier achieved the highest overall performance, with 97.79% accuracy, 97.82% precision, 97.79% recall, and 97.80% F1-score, followed by the Support Vector Machine, which achieved 93.30% accuracy and 93.28% F1-score. Among deep learning models, BiLSTM and BiGRU demonstrated competitive performance in capturing contextual patterns of abusive language. Class wise analysis showed that ethnicity and age were classified most accurately by the strongest traditional models, whereas gender remained the most challenging category across both machine learning and deep learning approaches.
Discussion:
The findings highlight the complementary strengths of traditional machine learning in identifying explicit lexical patterns and deep learning in capturing contextual representations of abusive language. Furthermore, variations in category specific performance demonstrate the importance of evaluating cyberbullying detection systems beyond aggregate metrics. This study provides insights into developing more robust, equitable, and culturally sensitive automated content moderation systems.
