跨语言稀疏的MoE蒸,以实现高效的低资源的阿萨姆语-英语和博多语-英语翻译
1School of Computer Science Engineering and Technology, Bennett University, Greater Noida, Uttar Pradesh, 201310, India.
Scientific reports
|January 13, 2026
概括
我们为阿萨米语和博多语等资源较少的语言开发了一种高效的神经机器翻译模型. 这种新模式使用稀疏的专家混合 (MoE) 和知识蒸,实现高翻译质量,显著减少在设备上部署的计算.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 计算语言学 计算语言学
背景情况:
- 像mBART50这样的大型多语言模型为低资源语言提供了改进的神经机器翻译 (NMT),但对于在设备上使用来说太大了.
- 阿萨姆语和博多语,尽管不是mBART50的预培训语言,但受益于跨语言适应.
研究的目的:
- 为阿萨姆语和博多语开发一个计算高效的NMT模型,保持高翻译质量.
- 为低资源语言实现实时,设备上的NMT.
主要方法:
- 微调的mBART50在阿萨姆语-英语和博多语-英语语体上进行了跨语言适应.
- 提出了一种两阶段的方法,将稀疏的专家混合 (MoE) 架构与跨语言知识蒸相结合.
- 实现了一种12层的变压器编码解码器,配有MoE的前块和语言前嵌入.
主要成果:
- 实现了4亿个参数的学生模型,翻译质量接近 (在1个BLEU点内) 与13亿个参数的教师模型.
- 与教师模型相比,每令牌的活跃计算减少了大约四倍.
- 在RTX 3050 GPU上,每个句子的演速度约为24ms,比密集的老师快280%.
结论:
- 拟议的基于MoE的跨语言蒸有效地实现了在边缘的高效,高质量的Indic NMT.
- 这种方法可以显著降低模型大小和计算成本,同时保持翻译准确度.
- 为低资源语言的实际设备翻译铺平了道路.
相关概念视频
02:07Improving Translational Accuracy
3.5K
3.5K
02:07Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
01:31Translation
155.2K
Lesson: Translation
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...
155.2K
01:31Translation
17.5K
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Proteins are...
Translation Produces the Building Blocks of Life
Proteins are...
17.5K
02:28Leaky Scanning
5.6K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.6K
02:33Initiation of Translation
7.9K
7.9K