Back-translation effects on static and contextual word embeddings for topic classification embedding in...

Area of Science:

• Natural Language Processing
• Machine Learning

Background:

• Topic classification is crucial for organizing text data.
• Data augmentation techniques like back-translation can enhance model performance.
• Evaluating these techniques across different word embedding types is essential.

Purpose of the Study:

• To assess the impact of back-translation on topic classification performance.
• To compare its effectiveness on static word vectors (FastText) versus contextual embeddings (RoBERTa).
• To determine benefits for low-resource languages and various classification algorithms.

Main Methods:

• Experiments utilized Logistic Regression, SVM, Random Forest, and RNN-LSTM classifiers.
• Datasets were augmented using back-translation across six languages.
• Performance was evaluated using F1-scores, comparing original and augmented data.

Main Results:

• Back-translation consistently improved F1-scores for static embeddings (up to 2.80% for Random Forest).
• Improvements for RNN-LSTM with static embeddings were smaller and often not statistically significant.
• Back-translation had a negligible impact on RoBERTa's contextual embeddings, showing no significant F1-score gains.

Conclusions:

• Back-translation is a valuable data augmentation strategy for static word embeddings in topic classification, particularly for low-resource languages.
• Modern contextual models like RoBERTa demonstrate high performance with less reliance on external augmentation.
• The utility of back-translation is limited for advanced context-aware models.