Advanced natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Large Language Models (LLMs) require adaptation for specific languages.
• Meta-AI's LLaMA-3 model provides a strong foundation for LLM development.
• Instruction tuning and preference optimization are key for enhancing LLM capabilities.

Purpose of the Study:

• To develop an LLM optimized for the Italian language.
• To improve instruction-following and reduce biases in LLMs.
• To achieve state-of-the-art performance on Italian language tasks.

Main Methods:

• Fine-tuning an 8B parameter instruction-tuned model using Supervised Fine-tuning (SFT) on English datasets.
• Applying Direct Preference Optimization (DPO) to align preferences and mitigate unsafe responses.
• Adapting the model to Italian using high-quality Italian data and QLoRA for computational efficiency.

Main Results:

• LLaMAntino-3-ANITA-8B-Inst-DPO-ITA demonstrates state-of-the-art performance among Italian LLMs.
• Achieved an average accuracy score of 0.6160 on Italian Open LLM benchmarks.
• Effective in Italian text comprehension and question-answering tasks.

Conclusions:

• The developed LLM is highly effective for Italian language processing.
• The methodology combines SFT, DPO, and QLoRA for efficient and high-performing model adaptation.
• The model is publicly available on HuggingFace, promoting further research and application.