STYLETTS-VC:通过知识转移从基于风格的TTS模型进行一次性声音转换
Yinghao Aaron Li1, Cong Han1, Nima Mesgarani1
1Department of Electrical Engineering, Columbia University, USA.
概括
本研究介绍了一种使用从文本到语音模型的转移学习进行一次性语音转换 (VC) 的新方法. 该方法实现了高度现实的语音转换,不需要输入文本,优于现有的方法.
科学领域:
- 人工智能的人工智能
- 语音处理 语音处理
- 机器学习 机器学习
背景情况:
- 一次性语音转换 (VC) 能够使用最小的目标音箱数据进行语音风格传输.
- 从语音内容中解脱发言者的身份是VC的一个关键挑战.
- 现有的方法往往需要大量的训练数据或与自然性作斗争.
研究的目的:
- 开发一种用于一次性语音转换 (VC) 的新方法.
- 为了实现高保真语音传输的脱而出的语音表示.
- 为了提高转换话语的自然性和相似性.
主要方法:
- 利用基于风格的文本到语音 (TTS) 模型的转移学习.
- 采用循环一致和对抗性培训用于转录引导的VC.
- 实施教师-学生知识传输,使用mel-spectrogram编码器和数据增强来实现文本独立表示.
主要成果:
- 提出的方法成功地解开了演讲者的身份和演讲内容.
- 在一次性语音转换中实现了高保真性和相似性.
- 在对自然性和相似性的主观评价中显著超过了最先进的模型.
结论:
- 这种新的方法有效地解决了在一次性VC中脱而出的语音表示的挑战.
- 从TTS模型转移学习为高级语音转换提供了一个有希望的方向.
- 文本独立的方法提高了一次性VC系统的实用性和性能.
相关概念视频
Gene Conversion
2.4K
2.4K
Improving Translational Accuracy
2.6K
2.6K
Language Development
395
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
395
Source Transformation
6.6K
Source transformation is a fundamental technique employed in circuit analysis, offering a valuable tool for simplifying complex electrical circuits. This technique involves the replacement of either a voltage source in series with a resistor by a current source in parallel with a resistor, or vice versa. The key concept here is that when the original sources are deactivated (turned off), the equivalent resistance at the circuit's end terminals remains the same.
It is essential to note that when...
It is essential to note that when...
6.6K
Air-entraining Agents
95
Air-entraining agents improve the durability and workability of concrete in climates with frequent freezing and thawing. These agents prevent cracks by introducing small air bubbles into the mix, creating spaces accommodating water expansion when temperatures drop. The air-entraining agents lower the surface tension of water, forming stable, small air bubbles. This method is more effective than having accidental large voids, as the intentional, smaller, and evenly distributed air voids improve...
95
Conversion of Units
22.9K
Sometimes, there is a need to convert from one unit to another one. For instance, reading a cookbook in which quantities are expressed in units of liters or ounces may require conversion of quantities to cups. Or, when looking up directions on how to get to a location, we may be interested to know how many miles we are going to walk. In this case, we would have to convert units of feet or meters to miles.
The first step in the unit conversion is to list the given units and the units required...
The first step in the unit conversion is to list the given units and the units required...
22.9K


