Multi-Ideology, Multiclass Online Extremism Dataset, and Its Evaluation Using Machine Learning

Area of Science:

• Computer Science
• Social Sciences
• Computational Linguistics

Background:

• Social media platforms are exploited for extremist propaganda, radicalization, and recruitment, necessitating effective detection methods.
• Existing extremism detection research is limited by single-ideology datasets, binary classification, and manual data validation.
• Challenges include class imbalance and a lack of automated data validation in current extremism detection studies.

Purpose of the Study:

• To develop a balanced, multi-ideology extremism text dataset for improved detection.
• To classify extremism into propaganda, radicalization, and recruitment categories using robust validation.
• To address limitations of existing single-ideology and binary classification datasets.

Main Methods:

• Created a versatile extremism text dataset generalizing multiple ideologies (ISIS, White Supremacist).
• Utilized TF-IDF (unigram, bigrams, trigrams) and pretrained word2vec features for analysis.
• Evaluated machine learning classifiers including Naïve Bayes, Support Vector Machine, Random Forest, and XGBoost.

Main Results:

• The Support Vector Machine with TF-IDF unigram features achieved the best performance with a 0.67 F1 score.
• The proposed multi-ideology, multi-class dataset demonstrated comparable performance to existing single-ideology datasets.
• Feature extraction included TF-IDF and word2vec for semantic analysis.

Conclusions:

• The developed multi-ideology dataset enhances extremism text classification accuracy.
• Machine learning models can effectively identify propaganda, radicalization, and recruitment on social media.
• This research contributes a valuable resource for combating online extremism through improved detection.