Fusing Transformer-XL with bi-directional recurrent networks for cyberbullying detection.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Social Media Analytics

Background:

• Cyberbullying detection is challenging in non-English languages due to linguistic nuances and data scarcity.
• Existing methods often struggle with the complexities of low-resource languages like Bengali.

Purpose of the Study:

• To develop and evaluate a robust method for identifying cyberbullying in Bengali text data.
• To address the limitations of current cyberbullying detection systems in under-resourced linguistic contexts.

Main Methods:

• A hybrid model combining Transformer-XL with BiGRU-BiLSTM (Fusion Transformer-XL) was developed.
• Extensive data preprocessing, including cleaning, augmentation, and handling imbalanced classes, was performed.
• Tokenization using a pre-trained model and Local Interpretable Model-Agnostic Explanations (LIME) were employed for interpretability.

Main Results:

• The Fusion Transformer-XL model achieved a high accuracy of 98.17% and an F1-score of 98.18%.
• The model demonstrated superior performance compared to baseline models.
• Cross-dataset evaluation and k-fold cross-validation confirmed the model's robustness and adaptability.

Conclusions:

• The proposed Fusion Transformer-XL model is effective for Bengali cyberbullying detection, highlighting the power of hybrid architectures.
• This research contributes to advancing cyberbullying detection methods for languages with limited resources.
• The study opens avenues for further research in multilingual NLP and social media analysis.