TELEIA: A Spanish language dataset for evaluating artificial intelligence models.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Existing Large Language Model (LLM) evaluations often rely on translated English tests.
• These translations may not accurately assess Spanish language proficiency.
• There is a need for dedicated Spanish language evaluation datasets for LLMs.

Purpose of the Study:

• To introduce TELEIA, a novel dataset for evaluating Spanish language knowledge in Large Language Models (LLMs).
• To provide a resource that complements existing English-centric LLM evaluation tools.
• To facilitate more accurate and nuanced assessment of LLMs' Spanish language capabilities.

Main Methods:

• Development of a multiple-choice question dataset (TELEIA) specifically designed for Spanish language evaluation.
• Questions are structured to match the format and difficulty of human Spanish proficiency tests.
• The dataset includes 100 questions prepared and validated by Spanish language experts.

Main Results:

• TELEIA provides a specialized tool for assessing LLMs' understanding of Spanish.
• The dataset's multiple-choice format allows for automated testing and integration into LLM Leaderboards.
• TELEIA aims to enhance the evaluation of LLMs in Spanish language tasks.

Conclusions:

• TELEIA represents a significant step towards more accurate evaluation of LLMs in Spanish.
• The dataset will be incorporated into the first Leaderboard for Spanish LLMs, promoting standardized assessment.
• This resource is crucial for advancing LLM development and application in the Spanish-speaking world.