在智能空间中对多个扬声器进行视听跟踪
Frank Sanabria-Macias1, Marta Marron-Romera1, Javier Macias-Guarasa1
1Universidad de Alcalá, Department of Electronics, Engineering School, Campus Universitario, 28805 Alcalá de Henares, Spain.
Sensors (Basel, Switzerland)
|August 12, 2023
概括
这项研究介绍了GAVT,GAVT是一种视听3D追踪系统,可以使用单一的摄像头和麦克风阵列精确定位扬声器. 与只有音频或只有视频的方法相比,GAVT显著提高了跟踪精度.
科学领域:
- 计算机视觉 计算机视觉
- 音频信号处理 音频信号处理
- 机器人技术 机器人技术 机器人技术
背景情况:
- 准确的扬声器定位对于人机交互和辅助技术至关重要.
- 现有的视听追踪系统面临着阻塞和多语音环境的挑战.
研究的目的:
- 开发一个强大的视听3D追踪系统 (GAVT),用于精确的扬声器定位.
- 通过整合先进的视觉和音频处理技术来提高追踪精度.
- 有效地处理闭塞并区分多个扬声器.
主要方法:
- 使用颗粒过器和视听3D跟踪的概率框架.
- 开发了一个复杂的视觉外观模型,将Viola & Jones面部检测器内核转换为概率估计器.
- 实施了一种基于概率分散的阻塞处理机制.
- 采用概率式定向响应功率用于音频本地化,将交叉相关函数建模为高斯混合模型.
- 引入了一种新的高斯 - 扬声器关联机制,以防止跟踪器干扰.
主要成果:
- 与仅视频模式相比,GAVT在3D本地化方面取得了显著的改善 (相对改善高达50.3%).
- 在2D (AV16.3数据集) 和18.1%的3D (CAV3D数据集) 中,单个和多个对象跟踪任务的平均相对改善达到了69.7%,相比于最先进的状态.
- 在AV16.3和CAV3D数据库中成功评估了SOT和MOT任务.
结论:
- GAVT提供了一个高度准确的视听3D追踪解决方案.
- 该系统有效地解决了诸如闭塞和多音箱场景等挑战.
- GAVT代表了扬声器本地化技术的重大进步.
相关概念视频
Perceiving Loudness, Pitch, and Location
239
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
239
Auditory Perception
364
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
364
Perception of Sound Waves
4.5K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.5K
Auditory Pathway
5.5K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
5.5K
Distance Measurements by Taping
66
Tapes are essential in surveying for accurate, durable, and short-distance measurements. Made from lightweight, nylon-coated steel, they offer flexibility and strength for rugged outdoor use. The nylon coating protects against rust and wear, extending the tape's life. Standard lengths, around 30 meters, are marked in meters and millimeters for precision.Surveyors select tapes based on site conditions and accuracy needs. Lightweight, nylon-coated tapes are commonly used for ease of handling and...
66
Sound Waves: Interference
3.8K
Sound waves can be modeled either as longitudinal waves, wherein the molecules of the medium oscillate around an equilibrium position, or as pressure waves. When two identical waves from the same source superimpose on each other, the combination of two crests or two troughs results in amplitude reinforcement known as constructive interference. If two identical waves, that are initially in phase, become out of phase because of different path lengths, the combination of crests with troughs...
3.8K


