有效的零射击多扬声器文本到语音技术使用信息扰乱和扬声器编码器
Chae-Woon Bang1,2, Chanjun Chun1,2
1Department of Computer Engineering, Chosun University, Gwangju 61452, Republic of Korea.
Sensors (Basel, Switzerland)
|December 9, 2023
概括
这项研究引入了一种改进的Grad-TTS模型,用于零射击的多扬声器语音合成. 这种新的方法可以通过利用来自引用的扬声器信息来为未见的扬声器生成高质量的语音.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 自然语言处理自然语言处理.
背景情况:
- 深度学习显著提高了语音合成质量.
- 扩散模型Grad-TTS提供高质量的多扬声器合成,但与看不见的扬声器作斗争.
研究的目的:
- 开发一个有效的零射击多扬声器语音合成模型.
- 为了使在培训数据中不存在的演讲者能够进行语音合成.
主要方法:
- 提出了一个增强的Grad-TTS架构.
- 一个预训练的扬声器识别模型从引用中提取扬声器信息.
- 信息扰动允许学习各种扬声器特征.
主要成果:
- 该模型在视听扬声器方面表现出色.
- 与现有的零射击模型相比,对于未见的扬声器实现了相似的性能.
- 客观指标,如扬声器编码器共相似度 (SECS) 和平均意见得分 (MOS) 被用于评估.
结论:
- 拟议的方法有效地扩展了Grad-TTS,用于零射击的多扬声器语音合成.
- 这种方法成功地为看不见的扬声器合成了语音,克服了以前的局限性.
相关概念视频
Interference: Path Lengths
1.3K
Consider two sources of sound, that may or may not be in phase, emitting waves at a single frequency, and consider the frequencies to be the same.
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...
1.3K
Reconstruction of Signal using Interpolation
203
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
203
Design Example
331
The innovation of touch-tone telephony revolutionized the telecommunications industry by replacing the traditional rotary dial with a dual-tone multi-frequency (DTMF) signaling system. This system uses a matrix-style keypad with buttons arranged in four rows and three columns, creating 12 distinct signals each assigned to a pair of frequencies. Each button press results in a simultaneous generation of two sinusoidal tones – one from a low-frequency group (697 to 941 Hz) and one from a...
331
Sound Waves: Interference
3.8K
Sound waves can be modeled either as longitudinal waves, wherein the molecules of the medium oscillate around an equilibrium position, or as pressure waves. When two identical waves from the same source superimpose on each other, the combination of two crests or two troughs results in amplitude reinforcement known as constructive interference. If two identical waves, that are initially in phase, become out of phase because of different path lengths, the combination of crests with troughs...
3.8K


