通过知识蒸进行零射击测试时间的调整,以实现个性化的语音消音和脱声
Sunwoo Kim1, Mrudula Athi1, Guangji Shi1
1Amazon Lab126, Sunnyvale, California 94089, USA.
The Journal of the Acoustical Society of America
|February 16, 2024
概括
一个新的框架为特定环境定制了紧的语音增强 (SE) 模型. 这种零射击方法在没有清洁语音数据的情况下提高了性能,超过了较大的模型.
科学领域:
- 语音处理 语音处理
- 机器学习 机器学习
- 声学 声学 在声学方面
背景情况:
- 紧型号对于设备上的语音增强 (SE) 是必不可少的.
- 将SE模型适应特定的噪音和反响环境是一项挑战.
- 数据稀缺,特别是清洁的语言,阻碍了个性化.
研究的目的:
- 为紧的SE模型提出个性化框架.
- 为了在特定的,反复出现的声学条件下提高SE性能.
- 为了应对无法提供培训的清洁语言数据的挑战.
主要方法:
- 使用知识蒸的零射击个性化方法.
- 一个大型的"老师"模型为训练一个紧的"学生"模型提供伪目标.
- 这种方法可以避免在个性化过程中需要清洁的语音目标.
主要成果:
- 个性化显著提高了紧的学生模型的性能.
- 个性化模型的表现优于更大,非个性化的基线模型.
- 实现了模型压缩,在脱声和消声方面没有损失.
结论:
- 拟议的框架有效地个性化了紧的SE模型.
- 零射击个性化是数据稀缺场景的可行解决方案.
- 个性化可以在边缘设备上实现高效,高性能的SE.
相关概念视频
Downsampling
157
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
157
Difference from Background: Limit of Detection
6.4K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
6.4K
Perceiving Loudness, Pitch, and Location
212
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
212
Generalization, Discrimination, and Extinction
557
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
557
Chunking and Rehearsal in Sensory Memory
212
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
212
Sampling Continuous Time Signal
243
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
243


