基于YOLO和TDOA的会议中的自动扬声器定位
Chen-Chiung Hsieh1, Men-Ru Lu1, Hsiao-Ting Tseng2
1Department of Computer Science and Engineering, Tatung University, Taipei City 104, Taiwan.
Sensors (Basel, Switzerland)
|July 29, 2023
概括
本研究介绍了一种新的扬声器本地化方法,用于使用YOLO对象检测和到达时间差 (TDOA) 音频本地化进行视频会议. 综合方法实现了高精度,改进了单一来源方法.
科学领域:
- 计算机视觉 计算机视觉
- 声学信号处理 声学信号处理
- 人与计算机的交互
背景情况:
- 视频会议在COVID-19大流行期间变得普遍,突出了标准网络摄像机在跟踪演讲者的局限性.
- 目前的网络摄像头技术缺乏自动翻/倾和扬声器锁定功能,阻碍了动态交互.
- 有效的演讲者识别对于增强虚拟沟通和会议参与度至关重要.
研究的目的:
- 开发一个用于视频会议中实时定位演讲者的自动化系统.
- 整合视觉 (物体检测) 和听觉 (声音源定位) 线索,以精确跟踪扬声器.
- 评估拟议的组合方法与单个技术的性能.
主要方法:
- 利用YOLO (You Only Look Once) 对象检测来识别个人并检测口腔运动 (说话指标).
- 使用到达时间差异 (TDOA) 带有麦克风阵列来确定声音源的角度.
- 结合YOLO衍生空间坐标与TDOA角度,以使用逆三角学准确定位扬声器.
主要成果:
- YOLOX-Tiny实现了85.2%的定位回忆率;仅TDOA的回忆率达到88%.
- 综合的YOLOX-Tiny和TDOA方法的回忆率为86.7%,精度为100%,准确率为94.5%.
- 综合方法表现出优异的性能,而不是使用视觉或听觉定位独立使用.
结论:
- 拟议的方法有效地定位了视频会议设置中的主动演讲者.
- 整合YOLO物体检测和TDOA显著提高了扬声器定位的准确性和可靠性.
- 这项技术为改善虚拟会议中的互动性和用户体验提供了强大的解决方案.
相关概念视频
01:18Distance Measurements by Taping
66
Tapes are essential in surveying for accurate, durable, and short-distance measurements. Made from lightweight, nylon-coated steel, they offer flexibility and strength for rugged outdoor use. The nylon coating protects against rust and wear, extending the tape's life. Standard lengths, around 30 meters, are marked in meters and millimeters for precision.Surveyors select tapes based on site conditions and accuracy needs. Lightweight, nylon-coated tapes are commonly used for ease of handling and...
66
01:30Design Example: Identifying the Locations of Monuments in the Field Using Global Positioning System Device
78
Surveyors use Global Positioning System (GPS) technology to measure the precise location and elevation of points on Earth. In a recent survey, GPS receivers were used to determine the coordinates and elevations of two park monuments. The process involved careful mission planning, data collection, and correction to ensure accuracy. The survey began with mission planning to identify optimal satellite visibility and minimize Position Dilution of Precision (PDOP). A geodetic control point...
78
01:28Field Application of Global Positioning System
68
The Global Positioning System (GPS) has become an indispensable tool in fieldwork, offering unparalleled precision and efficiency for surveying, navigation, and infrastructure development. By harnessing signals from a constellation of satellites, GPS receivers determine the location of objects with remarkable speed and accuracy, often completing calculations within a second.Advantages of Modern GPS TechnologyContemporary GPS receivers are designed to meet the practical demands of field...
68
01:21Perceiving Loudness, Pitch, and Location
239
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
239
01:10Design Example: Measuring Distance Between Two Points with Obstructions
65
When measuring distances in areas with physical obstructions, such as a lake in a field, surveyors must employ techniques to calculate accurate lengths without direct line measurements. One effective method is the offset technique, which allows for precise distance estimation over inaccessible stretches.In this scenario, a surveyor must measure a side of an area that crosses a lake. Since the measuring tape cannot span the lake, the surveyor begins by establishing a baseline that aligns with...
65
01:21Linear Approximation in Time Domain
101
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
101