Investigating cross-lingual training for offensive language detection.

Area of Science:

• Computational Linguistics
• Natural Language Processing
• Machine Learning

Background:

• User-generated content platforms require robust offensive speech detection.
• Existing automatic methods often lack effectiveness in low-resource languages due to limited labeled datasets.
• Cross-lingual transfer learning shows promise but has yielded suboptimal results.

Purpose of the Study:

• To investigate the performance drop in cross-lingual transfer learning for offensive speech detection.
• To systematically compare pre-trained models and intermediate training strategies across five languages.
• To identify factors contributing to performance gains and common errors in multilingual content moderation.

Main Methods:

• Systematic comparison of various pre-trained language models.
• Evaluation of different intermediate training regimes.
• Analysis of classifier confidence and language model vocabulary on five target languages.

Main Results:

• Utilizing superior pre-trained language models led to substantial improvements in overall performance and zero-shot transfer.
• Intermediate training on related languages proved effective when target-language data was scarce.
• Analysis revealed insights into performance gains and typical error sources.

Conclusions:

• The choice of pre-trained language model is critical for effective cross-lingual transfer in offensive speech detection.
• Intermediate training offers a viable strategy for enhancing model performance with limited target data.
• Understanding model confidence and vocabulary aids in refining multilingual content moderation systems.