Asian hate speech detection on Twitter during COVID-19.

Area of Science:

• Natural Language Processing
• Computational Social Science
• Hate Speech Detection

Background:

• The COVID-19 pandemic exacerbated anti-Asian racism, particularly in Western countries.
• Official data indicates a significant increase in anti-Asian hate crimes during 2020.
• Social media platforms like Twitter became a space for both expressing and potentially identifying such hate speech.

Purpose of the Study:

• To establish a baseline of anti-Asian hate crimes on Twitter.
• To develop and evaluate machine learning models for classifying hate speech in tweets.
• To improve the performance of hate speech detection models by addressing dataset bias.

Main Methods:

• Collected a large corpus of tweets using the Twitter API V-2.
• Annotated a subset of 3,000 tweets for hate speech by four annotators.
• Applied various machine learning (ML) and deep learning (DL) models, including Logistic Regression, Random Forest, LSTM, and BERT.
• Balanced the dataset and filtered ambiguous tweets based on Fleiss Kappa agreement.

Main Results:

• Logistic Regression achieved the best performance among statistical ML models with an F1 score of 0.72.
• BERT, a deep learning model, demonstrated superior performance with an F1 score of 0.85.
• Data balancing and filtering techniques improved model performance in hate speech classification.

Conclusions:

• Deep learning models, particularly BERT, are highly effective for detecting anti-Asian hate speech on Twitter.
• Machine learning approaches can be valuable tools in monitoring and combating online hate speech.
• Further research will utilize the full 10 million tweet dataset for more comprehensive analysis.