Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Global digital communication necessitates multilingual user sentiment analysis.
• Existing methods struggle with language independence in opinion mining and social media monitoring.
• Transformer models offer advanced capabilities for text understanding.

Purpose of the Study:

• To develop and evaluate a language-independent sentiment analysis framework.
• To compare prompt-based fine-tuning with classical and deep learning approaches.
• To assess the performance of multilingual transformer models across diverse languages.

Main Methods:

• Implemented classical machine learning (SVM, Logistic Regression) with TF-IDF.
• Developed a hybrid deep learning model combining LSTM and CNNs.
• Fine-tuned multilingual transformer models (BERT-base-multilingual, XLM-RoBERTa) using prefix and cloze-style prompts for language-independent sentiment classification.

Main Results:

• XLM-RoBERTa with prompt-based fine-tuning significantly outperformed classical and deep learning methods.
• Prefix prompts achieved performance comparable to standard fine-tuning using only 32 training examples per class.
• The unified framework demonstrated effectiveness across eight typologically diverse languages.

Conclusions:

• Prompt-based fine-tuning is a highly effective strategy for scalable, language-independent sentiment analysis.
• Transformer models, especially XLM-RoBERTa, show great promise for cross-lingual sentiment classification.
• This approach reduces data requirements for training effective multilingual sentiment analysis models.