Detecting racism and xenophobia using deep learning models on Twitter data: CNN, LSTM and BERT

Area of Science:

• Computational Linguistics
• Artificial Intelligence
• Social Media Analysis

Background:

• Manual content moderation on social media is infeasible due to rapid growth.
• Natural Language Processing (NLP) enables automated text classification.
• Existing NLP models often lack sufficient training data for specific languages like Spanish.

Purpose of the Study:

• To develop and compare BERT-based and other deep learning models for detecting racist and xenophobic messages in Spanish tweets.
• To evaluate the performance of models trained on Spanish-specific data versus multilingual data.
• To underscore the importance of native transfer learning models for Spanish NLP tasks.

Main Methods:

• Development of five predictive models: two BERT-based (BETO, mBERT) and three deep learning models (CNN, LSTM, CNN+LSTM).
• Training and evaluation of models using Spanish tweets containing racist and xenophobic content.
• Comparative analysis of model performance based on precision metrics.

Main Results:

• The BETO model, a BERT variant trained exclusively on Spanish text, achieved the highest precision at 85.22%.
• The mBERT model achieved 82.00% precision, outperforming other deep learning models.
• CNN, LSTM, and CNN+LSTM models showed precision ranging from 79.34% to 80.48%.

Conclusions:

• Native transfer learning models, like BETO, are crucial for effectively addressing NLP challenges in Spanish.
• The study demonstrates the superior performance of Spanish-specific models in detecting hate speech.
• Findings support the development of tailored NLP solutions for diverse linguistic contexts.