Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages

Area of Science:

• Natural Language Processing (NLP)
• Computational Linguistics

Background:

• Advanced language models rely heavily on linguistic resources.
• Low-resource languages, such as Azerbaijani, face challenges due to limited labeled datasets, hindering effective model training.

Purpose of the Study:

• To enhance the effectiveness and generalization capabilities of news text classification models for low-resource languages.
• To address the scarcity of labeled data in Azerbaijani NLP by employing text augmentation.

Main Methods:

• Utilized text augmentation techniques for news text classification.
• Employed translation methods using Facebook's mBart50 model and the Google Translate API.
• Combined mBart50 and Google Translate for expanded text processing capabilities.

Main Results:

• Models trained on augmented data showed improved classification performance compared to those trained on original data.
• Demonstrated the potential of combined data augmentation strategies for underrepresented languages.
• Published a labeled text classification dataset and a pre-trained RoBERTa model for Azerbaijani.

Conclusions:

• Text augmentation, particularly using combined translation strategies, significantly enhances NLP model performance for low-resource languages.
• The developed resources (dataset and RoBERTa model) will aid future research and development in Azerbaijani NLP.