在线二元语音分离移动的扬声器与一个波网络
1Department of Electrical Engineering, Columbia University, New York, NY.
概括
这项研究引入了一个实时双耳语音分离模型,以解决移动扬声器的扬声器交换问题. 这种新的方法有效地跟踪扬声器,提高了长形式音频分离的一致性.
科学领域:
- 音频信号处理 音频信号处理
- 机器学习用于语音
背景情况:
- 现实世界语音分离通常涉及移动扬声器,使音频处理复杂化.
- 目前的方法,比如发言级别交替不变训练 (u-PIT),与时间输出一致性作斗争,导致扬声器交换.
- 扬声器交换问题由不断移动的扬声器加剧,挑战一致的频道配置.
研究的目的:
- 开发一个实时双声语音分离模型,减轻移动扬声器的扬声器交换问题.
- 为了确保长形式的音频分离中一致的扬声器跟踪和输出通道放置.
主要方法:
- 开发了一个基于波分网络的实时双耳语音分离模型.
- 该模型计算了每个时间框架的扬声器嵌入,使用在线聚类将它们汇总起来,并使用中心体来跟踪扬声器.
- 这种方法可以在长时间的音频段中连续跟踪单个扬声器.
主要成果:
- 拟议的方法表明,在反响,长形式的移动多语者语音分离中,对扬声器交换的敏感性降低了.
- 性能与使用地面真相跟踪的基于u-PIT的模型相美.
- 保持了分离的准确性和音声间线索的保存.
结论:
- 开发的实时双声语音分离模型有效地解决了移动扬声器的扬声器交换问题.
- 在线聚类和扬声器分析方法确保在长时间内始终保持一致的跟踪和分离.
- 这种方法为现实世界的音频场景提供了强大的解决方案,具有动态扬声器运动.
相关概念视频
Interference: Path Lengths
1.3K
Consider two sources of sound, that may or may not be in phase, emitting waves at a single frequency, and consider the frequencies to be the same.
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...
1.3K
Sound Waves: Interference
3.8K
Sound waves can be modeled either as longitudinal waves, wherein the molecules of the medium oscillate around an equilibrium position, or as pressure waves. When two identical waves from the same source superimpose on each other, the combination of two crests or two troughs results in amplitude reinforcement known as constructive interference. If two identical waves, that are initially in phase, become out of phase because of different path lengths, the combination of crests with troughs...
3.8K
Propagation of Waves
2.4K
When a wave propagates from one medium to another, part of it may get reflected in the first medium, and part of it may get transmitted to the second medium. In such a case, the interface of the two mediums can be considered as a boundary that is neither fixed nor free.
Consider a scenario where a wave propagates from a string of low linear mass density to a string of high linear mass density. In such a case, the reflected wave is out of phase with respect to the incident wave, however the...
Consider a scenario where a wave propagates from a string of low linear mass density to a string of high linear mass density. In such a case, the reflected wave is out of phase with respect to the incident wave, however the...
2.4K
Perception of Sound Waves
4.5K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.5K


