相关实验视频

Updated: Jan 8, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

通过多阶段的知识蒸来提高端到端的语音翻译

Yue Zhou¹, Yuxuan Yuan¹, Yanyan Feng²

¹School of Informatics, Xiamen University, Xiamen, 361005, Fujian, China; Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism, China.

Neural networks : the official journal of the International Neural Network Society

|December 22, 2025

概括

本研究引入了多粒度知识蒸用于语音到文本翻译,改进了从教师模型转移知识的方法. 这种新方法通过专注于困难的翻译和调整跨模式表示来提高翻译质量.

科学领域:

人工智能的人工智能
自然语言处理自然语言处理.
机器学习机器学习

背景情况:

知识蒸 (KD) 通过机器翻译 (MT) 教师模型增强了语音到文本翻译 (ST).
当前的KD方法通过依赖教师输出分布转移有限的知识,阻碍学生模型的表现.
由于ST数据的稀缺性,进一步限制了从MT转移到ST模型的全面知识转移.

研究的目的:

提出一种多颗粒蒸方法,以便在ST中更有效地传递知识.
解决现有的KD方法在捕获深度表示和处理数据稀缺方面的局限性.
提高端到端的语音到文本翻译的整体质量和有效性.

主要方法:

引入了适应性词级蒸,以优先考虑具有挑战性的翻译.
实现了跨模态隐藏状态蒸,以对齐MT和ST模型表示,弥合语音-文本模式差距.
开发了一个多阶段知识蒸 (MSKD) 框架,利用外部ASR,MT和ST数据.

主要成果:

在 MuST-C 数据集上,MSKD 实现了最先进的性能,超过了之前的 KD 方法 +2.5 BLEU.
通过外部数据,MSKD超过了强大的端到端ST基线+2.9 BLEU和级联系统+1.9 BLEU.
在ST性能方面取得了显著的改进,突出了该方法的有效性和可扩展性.

结论:

拟议的多颗粒蒸方法显著提高了对ST的知识转移.
MSKD有效地利用各种数据源和蒸技术来逐步提高ST的性能.
该框架提供了一种可扩展和有效的方法,以推进端到端的语音到文本翻译能力.

关键词:

知识的蒸知识的蒸.多颗粒蒸的蒸方法多个阶段的蒸蒸.语音翻译语音翻译语音翻译

更多相关视频

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

相关概念视频

Improving Translational Accuracy

Improving Translational Accuracy

Improving Translational Accuracy

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

Impression Management Techniques IV: Altercasting

Impression Management Techniques IV: Altercasting

Altercasting is a strategic communication technique in which an individual imposes a specific identity or social role onto another person to influence their behavior and shape the interaction. By presuming a role—such as “responsible leader” or “patient person”—altercasting encourages the target to conform to that identity, often aligning their behavior with the expectations associated with the role. The power of this tactic lies in its subtlety; once a role...

Translation

Translation

Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Proteins are...

Translation

Translation

Lesson: Translation
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...

Termination of Translation

Termination of Translation

The large ribosomal subunit has several important structures essential to translation. These include the peptidyl transferase center (PTC) - which is the site where the peptide bond is formed - and a large, internal, water-filled tube through which the nascent polypeptide moves. This latter structure is called the Peptide Exit Tunnel, and it begins at the PTC and spans the body of the large ribosomal subunit. During translation, as the nascent polypeptide chain is synthesized, it passes through...