AMSunda: A novel dataset for Sundanese information retrieval

Area of Science:

• Natural Language Processing
• Information Science
• Computational Linguistics

Background:

• Information Retrieval (IR) is vital for search engines and databases.
• The Sundanese language corpus from Indonesia lacks sufficient data for IR tasks.
• Existing Sundanese datasets primarily support text classification and generation, not IR.

Purpose of the Study:

• Introduce AMSunda, the first dataset specifically for Sundanese Information Retrieval.
• Facilitate fine-tuning and evaluation of embedding models for the Sundanese language.
• Bridge the gap in resources for Sundanese NLP research.

Main Methods:

• Developed the AMSunda dataset using GPT-4o-mini LLM.
• Created 1499 documents, yielding 7492 triplet passages for model fine-tuning.
• Generated 7491 BEIR-compatible queries for evaluating retrieval performance.

Main Results:

• AMSunda provides triplet data for embedding model fine-tuning.
• AMSunda offers BEIR-compatible data for robust retrieval task evaluation.
• The dataset comprises 1499 documents, 7492 triplets, and 7491 queries.

Conclusions:

• AMSunda is a foundational resource for advancing Sundanese Information Retrieval.
• The dataset enables the development of specialized Sundanese embedding models.
• This work supports improved NLP applications for the Sundanese language.