On the effectiveness of limited-data large language model fine-tuning for Arabic

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Multilingual LLMs show promise in zero-shot and few-shot learning for Arabic NLP.
• Fine-tuned BERT variants currently achieve state-of-the-art (SOTA) performance on many Arabic tasks.
• Specialized models often require extensive annotated training data.

Purpose of the Study:

• To investigate the effectiveness of fine-tuning general-purpose LLMs for Arabic NLP tasks using minimal data.
• To compare the performance of fine-tuned LLMs against SOTA models on sentiment analysis, sarcasm detection, and news categorization.
• To analyze the impact of model size and data-efficient fine-tuning methods.

Main Methods:

• Fine-tuning GPT-4o mini and Gemma-3-27B on small subsets (3.0%-7.5%) of training data for Arabic NLP tasks.
• Evaluating performance on sentiment analysis (ArSAS), sarcasm detection (ArSarcasm), and news categorization (ASND).
• Comparing GPT-4o and GPT-4o mini to assess the effect of model size on fine-tuning efficiency and performance.

Main Results:

• Fine-tuning GPT-4o mini on small datasets surpassed previous SOTA results in sentiment analysis and sarcasm detection.
• Performance comparable to SOTA was achieved in news categorization with both GPT-4o mini and Gemma-3-27B.
• Larger models like GPT-4o required fewer labeled examples than smaller models (e.g., GPT-4o mini) for comparable performance.
• Predictable scaling laws were observed, allowing accurate performance estimation of larger models using smaller ones.

Conclusions:

• LLMs can be effectively adapted for Arabic NLP using data-efficient fine-tuning, outperforming established models without large annotated datasets.
• The proposed method demonstrates generalizability across different LLMs, including open-source options.
• Predictable scaling laws facilitate efficient model selection and adaptation strategies for Arabic NLP.