基于注意力的融合,用于复杂领域的骨导和空导语音提升
Heming Wang1, Xueliang Zhang2, DeLiang Wang1,3
1Department of Computer Science and Engineering, The Ohio State University, USA.
概括
这项研究介绍了一种新的语音增强方法,它结合了骨传导 (BC) 和空气传导 (AC) 麦克风. 这种先进的技术显著提高了语音清晰度,特别是在杂的环境中.
科学领域:
- 信号处理 信号处理
- 音频工程 音频工程
- 机器学习用于音频.
背景情况:
- 骨传导 (BC) 麦克风捕获头骨振动以发言,提供噪声免疫力,但带宽有限.
- 空调 (AC) 麦克风捕获全频段语音,但对声噪声非常敏感.
- 现有的方法很难有效地结合BC和AC信号,以获得最佳的语音增强.
研究的目的:
- 开发一个先进的语音增强系统,集成BC和AC麦克风信号.
- 为了利用BC和AC麦克风的互补优势,提高语音可理解性.
- 超越现有的单麦克风和混合语音增强技术.
主要方法:
- 利用卷积循环网络进行复杂的音频信号光谱映射.
- 实施了基于注意力的融合策略,包括早期融合和晚期融合,以整合BC和AC数据.
- 在各种噪音条件下训练和评估了语音信号模型.
主要成果:
- 提出的基于注意力的融合方法与其他BC-AC集成技术相比,表现优越.
- 观察到语音增强的显著改善,特别是在信号噪声比 (SNR) 低的环境中.
- 该方法在具有挑战性的声学场景中超过了传统的语音增强方法.
结论:
- 将BC和AC麦克风与卷积循环网络和基于注意力的融合相结合,对于语音增强非常有效.
- 拟议的方法提供了强大的性能,特别是在杂的环境中,传统方法失败.
- 这种混合麦克风策略在克服单个麦克风类型的局限性以实现清晰的言语方面取得了重大进展.
更多相关视频
03:49Author Spotlight: Optimizing EAS with Long Electrodes for Enhanced Cochlear Coverage and Hearing Preservation
Published on: October 11, 2024
674
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
374
相关概念视频
01:31Hearing
51.1K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
51.1K
01:12Double Resonance Techniques: Overview
145
Double resonance techniques in Nuclear Magnetic Resonance (NMR) spectroscopy involve the simultaneous application of two different frequencies or radiofrequency pulses to manipulate and observe two distinct nuclear spins. One important application of double resonance is spin decoupling, which selectively suppresses coupling with one type of nucleus while observing the NMR signal from another nucleus, simplifying the spectrum and enhancing resolution.
Spin decoupling is usually achieved by...
Spin decoupling is usually achieved by...
145
01:13The Cochlea
43.8K
The cochlea is a coiled structure in the inner ear that contains hair cells—the sensory receptors of the auditory system. Sound waves are transmitted to the cochlea by small bones attached to the eardrum called the ossicles, which vibrate the oval window that leads to the inner ear. This causes fluid in the chambers of the cochlea to move, vibrating the basilar membrane.
43.8K
01:01Perception of Sound Waves
4.4K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.4K
01:15Auditory Pathway
4.5K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
4.5K
02:16Resonance and Hybrid Structures
16.1K
According to the theory of resonance, if two or more Lewis structures with the same arrangement of atoms can be written for a molecule, ion, or radical, the actual distribution of electrons is an average of that shown by the various Lewis structures.
Resonance Structures and Resonance Hybrids
The Lewis structure of a nitrite anion (NO2−) may actually be drawn in two different ways, distinguished by the locations of the N–O and N=O bonds.
Resonance Structures and Resonance Hybrids
The Lewis structure of a nitrite anion (NO2−) may actually be drawn in two different ways, distinguished by the locations of the N–O and N=O bonds.
16.1K