Cross-dialectal Arabic translation: comparative analysis on large language models.

Area of Science:

• Natural Language Processing (NLP)
• Artificial Intelligence (AI)
• Computational Linguistics

Background:

• Arabic dialects present unique challenges in NLP due to linguistic variation.
• Large Language Models (LLMs) offer new possibilities for multilingual communication and text generation.

Purpose of the Study:

• To evaluate the performance of GPT-3.5, GPT-4, and Bard (Gemini) in translating Arabic dialects.
• To compare different prompting techniques (zero-shot and few-shot) for Arabic dialect translation.

Main Methods:

• GPT-3.5, GPT-4, and Bard (Gemini) were tested on QADI and MADAR datasets.
• Evaluation used metrics like cosine similarity, Sentence-BERT, TER, ROUGE, and BLEU.
• Zero-shot prompting was used for all dialects; few-shot prompting was used for Tunisian.

Main Results:

• GPT-4 significantly outperformed other models in translating Modern Standard Arabic (MSA) to Dialectal Arabic (DA) on the QADI dataset.
• GPT-4 demonstrated superior semantic similarity and sentence overlap identification.
• GPT-5 showed strong performance on the MADAR dataset, particularly in sentence overlap detection.

Conclusions:

• GPT-4 is a leading model for Arabic dialect translation, especially for MSA to DA.
• Prompting techniques like few-shot learning did not consistently improve translation performance.
• LLMs show promise for bridging communication gaps across diverse Arabic dialects.