无监督的多感官融合时空空间扬声器日记化
Paris Xylogiannis1, Nikolaos Vryzas1, Lazaros Vrysis1
1Multidisciplinary Media & Mediated Communication Research Group (M3C), Aristotle University, 54636 Thessaloniki, Greece.
Sensors (Basel, Switzerland)
|July 13, 2024
概括
这项研究通过将扬声器嵌入式与空间到达时间差异 (TDOA) 数据集成来增强扬声器日记化. 将这些特征结合起来,可以显著提高在会议录音中识别"谁什么时候说话"的准确性.
科学领域:
- 语音处理 语音处理
- 机器学习 机器学习
- 声学信号处理 声学信号处理
背景情况:
- 扬声器日记化可以识别音频录制中的扬声器.
- 来自麦克风阵列的空间特征可以帮助在会议中识别演讲者.
- 当前的方法通常仅依赖于音频功能,可能缺少空间线索.
研究的目的:
- 开发和评估一个框架,将扬声器嵌入与到达时间差 (TDOA) 结合起来,以改善扬声器日记化.
- 在与空间信息集成时,评估不同扬声器嵌入模型 (ECAPA-TDNN,X向量) 的有效性.
- 确定最佳方法,以组合时空信息,以集群发言人.
主要方法:
- 使用ECAPA-TDNN和X向量提取的扬声器嵌入.
- 抵达时间差 (TDOA) 计算使用一般化交叉相关性 (GCC) 与相变换 (PHAT) 权重.
- 在AVLab和SpeaD-M3C多通道数据集上评估框架,评估各种时空组合技术.
主要成果:
- 整合空间信息显著改善了最先进的深度学习日记化模型.
- 与基线方法相比,观察到日记错误率 (DER) 减少了2-3%.
- ECAPA-TDNN通常表现优于X向量,但两者都受益于空间数据集成.
结论:
- 将扬声器嵌入与空间TDOA数据相结合是提高扬声器日记化性能的有效策略.
- 空间信息提供了有价值的线索,可以补充基于音频的扬声器识别,特别是在会议环境中.
- 拟议的框架证明了利用多通道音频来实现更准确和更强大的扬声器识别的实用性.
相关概念视频
Perceiving Loudness, Pitch, and Location
205
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
205
Chunking and Rehearsal in Sensory Memory
198
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
198
Multi-input and Multi-variable systems
105
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
105


