乌兹别克-哈萨克斯坦机器翻译的平行文本数据集
Bobur Allaberdiev1, Gayrat Matlatipov2, Elmurod Kuriyozov2,3
1National University of Uzbekistan named after Mirzo Ulugbek, Universitet Street, 4, Olmazor district, 100174, Tashkent city, Uzbekistan.
Data in brief
|March 1, 2024
概括
为机器翻译创建了一个新的乌兹别克语和哈萨克语的平行语料库. 这个有价值的数据集,可在拥抱脸上,有助于研究低资源的土耳其语语言.
科学领域:
- 计算语言学 计算语言学
- 自然语言处理自然语言处理.
- 机器翻译 机器翻译
背景情况:
- 开发并行语料库对于推进机器翻译 (MT) 能力至关重要,特别是对于低资源语言.
- 乌兹别克语和哈萨克语的现有资源有限,这阻碍了为这些土耳其语语言开发有效的MT系统.
研究的目的:
- 为乌兹别克斯坦和哈萨克斯坦原始文本提供一种新的,高质量的并行库.
- 详细说明用于构建此数据集的方法.
- 突出该集体在各种自然语言处理 (NLP) 应用中重复使用的潜力,特别是机器翻译.
主要方法:
- 采用多阶段数据收集方法,结合现有并行数据,公开可用的资源 (文学,网络新闻) 和专家手册翻译.
- 应用了句子对齐技术来整合来自不同来源的数据.
- 最终的语料库被策划并通过拥抱面孔平台提供.
主要成果:
- 已经成功地创建和出版了大量的乌兹别克-哈萨克原始文本.
- 数据集涵盖了广泛的主题和类型,增加了它的实用性.
- 该库适用于训练和测试统计和神经机器翻译模型.
结论:
- 开发的乌兹别克斯坦-哈萨克斯坦并行语料库是对低资源土耳其语语言的NLP研究的重要贡献.
- 它在 Hugging Face 上的可用性促进了可访问性和协作开发.
- 这个资源可以作为一个基础模型,为其他相关语言创建类似的 corpora.
相关概念视频
02:07Improving Translational Accuracy
2.6K
2.6K
01:31Translation
14.9K
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Proteins are...
Translation Produces the Building Blocks of Life
Proteins are...
14.9K
02:04Polytene Chromosomes
3.0K
3.0K
01:16Crossover Experiments
2.8K
Crossover experiments, also called the repeated-measurements design, is a study design in which all experimental units are exposed to all treatments in different periods. Crossover experiments are generally used in psychology, the pharmaceutical industry, agriculture, and medicine.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
2.8K
01:13Multiple Comparison Tests
3.9K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.9K
01:23Test for Homogeneity
2.0K
The goodness–of–fit test can be used to decide whether a population fits a given distribution, but it will not suffice to decide whether two populations follow the same unknown distribution. A different test, called the test for homogeneity, can be used to conclude whether two populations have the same distribution. To calculate the test statistic for a test for homogeneity, follow the same procedure as with the test of independence. The hypotheses for the test for homogeneity can...
2.0K