3方向 Inception-ResUNet:深度空间特征学习,用于多通道唱歌声音分离与扭曲
DaDong Wang1, Jie Wang1, MingChen Sun2
1School of Mathematics and Computer Science, Jilin Normal University, Siping, Jilin, China.
PloS one
|January 29, 2024
概括
本研究介绍了一种新的3D Inception-ResUNet模型,用于机器人歌唱声音分离,通过利用空间和光谱信息显著提高性能. 多目标培训方法实现了平均11.55dBNSDR,优于现有方法.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 信号处理 信号处理
- 人工智能的人工智能
背景情况:
- 人形机器人在解释复杂的听觉信号方面面临挑战,例如混合的歌声,音乐和噪音.
- 机器人感知到的声信号往往会被扭曲,减弱和反响,使语音分离任务复杂化.
研究的目的:
- 为人形机器人开发一种先进的歌声分离模型.
- 为了提高机器人在杂环境中解释模两可的听觉信号的能力.
主要方法:
- 在U形网络中使用3D Inception-ResUNet架构来处理光谱图.
- 采用多目标培训,大小一致性损失,阶段一致性损失和大小相关性一致性损失.
- 通过NAO机器人和用于模型训练的MIR-1K数据集合成了10通道数据集.
主要成果:
- 拟议的模型在测试数据集上实现了平均11.55dB的规范化源与扭曲比率 (NSDR).
- 与现有的比较模型相比,在唱歌声音分离任务中表现出卓越的性能.
- 多目标培训战略有效地改善了空间和光谱信息的利用.
结论:
- 3D Inception-ResUNet模型在基于机器人的歌声分离方面取得了重大进展.
- 多目标培训对于提高机器人听觉信号解释的准确性和稳定性至关重要.
- 这项研究通过改进音频处理能力,为更复杂的人机交互做出了贡献.
相关概念视频
Perceiving Loudness, Pitch, and Location
212
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
212
Double Resonance Techniques: Overview
210
Double resonance techniques in Nuclear Magnetic Resonance (NMR) spectroscopy involve the simultaneous application of two different frequencies or radiofrequency pulses to manipulate and observe two distinct nuclear spins. One important application of double resonance is spin decoupling, which selectively suppresses coupling with one type of nucleus while observing the NMR signal from another nucleus, simplifying the spectrum and enhancing resolution.
Spin decoupling is usually achieved by...
Spin decoupling is usually achieved by...
210
Deconvolution
160
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
160
Linear Approximation in Frequency Domain
91
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
91


