Machine translation training data for English-Tshivenḓa

Area of Science:

• Computational Linguistics
• Natural Language Processing
• African Languages

Background:

• Machine translation (MT) systems require large, high-quality parallel corpora for effective training.
• The Tshivenḓa language, spoken in South Africa, has historically lacked sufficient digital resources for advanced NLP tasks.

Purpose of the Study:

• To describe the creation and characteristics of a novel parallel corpus for English-Tshivenḓa machine translation.
• To provide a valuable resource for developing Tshivenḓa MT systems and other language technologies.

Main Methods:

• Data collection through web crawling of South African government websites and matching documents from publishers.
• Augmentation of the corpus via professional translation of English data into Tshivenḓa.
• Detailed information on data alignment, corpus cleaning, and word count statistics.

Main Results:

• A substantial parallel corpus of English-Tshivenḓa data was compiled.
• The corpus includes both aligned parallel sentences and monolingual Tshivenḓa text.
• The data underwent rigorous cleaning and alignment processes.

Conclusions:

• The developed dataset is a significant contribution to Tshivenḓa language resources.
• This corpus can be utilized for training machine translation models and advancing Tshivenḓa core technologies.
• The data also supports linguistic research and the development of other Tshivenḓa NLP tools.