三级混合神经光束转换器用于多通道语音增强
Kelan Kuang1, Feiran Yang2, Junfeng Li3
1Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China.
The Journal of the Acoustical Society of America
|June 21, 2023
概括
这项研究介绍了TriU-Net,这是一种用于多通道语音增强的混合神经束转换器. 它通过明确解决语音扭曲补偿,显著提高了语音质量和可理解性.
科学领域:
- 信号处理 信号处理
- 人工智能的人工智能
- 声学 声学 在声学方面
背景情况:
- 多通道语音增强对于在杂环境中清晰的音频至关重要.
- 现有的方法经常与残余噪音和语音扭曲作斗争.
- 深度神经网络为先进的降噪和质量改进提供了潜力.
研究的目的:
- 提出一种新的混合神经束转换器,TriU-Net,用于增强的多通道语音处理.
- 在具有挑战性的声学条件下提高语音质量和可理解性.
- 引入一个有效的网络拓,以捕捉长距离的时间依赖.
主要方法:
- 一个三阶段的方法:以DNN估计的口罩进行最小差异无扭曲响应束形,以DNN为基础的噪声抑制后过,以及以DNN为基础的扭曲补偿.
- 利用一个封闭的卷积注意力网络来有效地表征远程时间依赖.
- 整合了显式语音扭曲补偿到增强管道中.
主要成果:
- 获得了2.854的平均宽带语音质量感知评估 (wb-PESQ) 评分.
- 在CHiME-3数据集上达到平均92.57%的扩展得分可理解性 (ESTOI).
- 通过对合成和现实世界数据的实验,在杂和反响环境中证明了有效性.
结论:
- 拟议的TriU-Net通过结合光束成形,后过和扭曲补偿,有效地增强了多通道语音.
- 明确考虑语音扭曲补偿导致优越的语音质量和可理解性.
- 封闭的卷积注意力网络对于模拟语音增强中的时间依赖是有效的.
相关概念视频
Cascaded Op Amps
678
Operational amplifiers (op-amps) are versatile electronic components that can be interconnected in a cascade - one after another in a linear sequence. This cascading is possible due to their infinite input resistance and zero output resistance, allowing them to maintain their input-output relationships even when connected in series.
In a cascaded system, each op-amp is referred to as a stage. The output of one stage drives the input of the subsequent stage. As the input signal passes through...
In a cascaded system, each op-amp is referred to as a stage. The output of one stage drives the input of the subsequent stage. As the input signal passes through...
678
Amplifying Signals via Enzymatic Cascade
8.5K
When a ligand binds to a cell-surface receptor, the receptor's intracellular domain changes shape, which may either activate its enzyme function or allow its binding to other molecules. The initial signal is amplified by most signal transduction pathways. This means that a single ligand molecule can activate multiple molecules of a downstream target. Proteins that relay a signal are most commonly phosphorylated at one or more sites, activating or inactivating the protein. Kinases catalyze...
8.5K
Reconstruction of Signal using Interpolation
254
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
254
Double Resonance Techniques: Overview
248
Double resonance techniques in Nuclear Magnetic Resonance (NMR) spectroscopy involve the simultaneous application of two different frequencies or radiofrequency pulses to manipulate and observe two distinct nuclear spins. One important application of double resonance is spin decoupling, which selectively suppresses coupling with one type of nucleus while observing the NMR signal from another nucleus, simplifying the spectrum and enhancing resolution.
Spin decoupling is usually achieved by...
Spin decoupling is usually achieved by...
248
Design Example: Vintage Mixing Console
258
A sound engineer at a music company recently encountered a problem. The output from their newly acquired studio's vintage mixing console was too low for the requirements of modern recording equipment. To rectify this situation, the engineer decided to design an audio pre-amplifier using an operational amplifier (op-amp) to boost the signal level.
The specifications for the pre-amplifier were clear. It needed to amplify the audio signal by a factor of 10, have an input impedance above 10...
The specifications for the pre-amplifier were clear. It needed to amplify the audio signal by a factor of 10, have an input impedance above 10...
258
Multi-input and Multi-variable systems
133
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
133


