Dataset for comparable evaluation of machine translation between 11 South African languages

Area of Science:

• Computational Linguistics
• Natural Language Processing
• African Languages

Background:

• Machine translation (MT) systems require robust evaluation datasets for development and benchmarking.
• Existing datasets may not adequately cover the linguistic diversity of South Africa's 11 official languages.
• The need for specialized parallel corpora to assess MT performance in low-resource language pairs is critical.

Purpose of the Study:

• To introduce and describe the Autshumato machine translation evaluation set.
• To provide a parallel dataset for evaluating machine translation systems across South Africa's official languages.
• To facilitate research and development in machine translation for African languages.

Main Methods:

• The creation of a parallel corpus containing sentences from 11 South African languages.
• Inclusion of four independent reference translations for each source sentence.
• The dataset covers Afrikaans, English, isiNdebele, isiXhosa, isiZulu, Sepedi, Sesotho, Setswana, Siswati, Tshivenḓa, and Xitsonga.

Main Results:

• A comprehensive parallel evaluation set for machine translation is now available.
• The dataset enables systematic evaluation of MT systems for various South African language pairs.
• Four reference translations per sentence enhance the reliability of MT system evaluations.

Conclusions:

• The Autshumato dataset is a significant contribution to the field of machine translation for African languages.
• It provides essential resources for improving the quality and performance of MT systems in South Africa.
• This evaluation set will foster further research and innovation in multilingual NLP technologies.