Fine-Tuning Arabic Large Language Models for improved multi-turn dialogue: A blueprint for synthetic data generation...

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• Arabic conversational systems face challenges due to limited large-scale, high-quality datasets for multi-turn dialogues.
• Existing datasets do not adequately capture the diversity required for robust Arabic dialogue models.

Purpose of the Study:

• To develop a reproducible methodology for constructing a large-scale, diverse Arabic multi-turn conversation dataset.
• To fine-tune and evaluate Arabic language models using this synthetic dataset.
• To establish a benchmark for Arabic conversational AI performance.

Main Methods:

• Structured prompting of an instruction-tuned Arabic LLM (Jais-13b-chat) to generate 43,316 multi-turn conversations.
• Fine-tuning two pre-trained Arabic models (ArabianGPT-08B-V2, AraGPT2-mega) on the synthetic dataset.
• Benchmarking using automatic metrics (Perplexity, RAVEN) and human evaluation.

Main Results:

• Fine-tuned ArabianGPT-08B-V2 achieved the highest RAVEN score (0.823), outperforming baselines.
• Maintained strong within-model perplexity (9.4) for ArabianGPT-08B-V2.
• Human evaluation showed positive quality scores (4.04-4.34) and acceptable inter-rater reliability.

Conclusions:

• LLM-generated synthetic data is effective for improving Arabic conversational models.
• The methodology provides a scalable and resource-efficient blueprint for dialogue systems in low-resource and culturally specific contexts.
• This work advances the development of sophisticated Arabic conversational AI.