Automatic Speaker Positioning in Meetings Based on YOLO and TDOA

Area of Science:

• Computer Vision
• Acoustic Signal Processing
• Human-Computer Interaction

Background:

• Video conferencing became prevalent during the COVID-19 pandemic, highlighting limitations of standard webcams in tracking speakers.
• Current webcam technology lacks automatic pan/tilt and speaker-locking capabilities, hindering dynamic interaction.
• Effective speaker identification is crucial for enhanced virtual communication and meeting engagement.

Purpose of the Study:

• To develop an automated system for real-time speaker localization in video conferences.
• To integrate visual (object detection) and auditory (sound source localization) cues for precise speaker tracking.
• To evaluate the performance of the proposed combined method against individual techniques.

Main Methods:

• Utilized YOLO (You Only Look Once) object detection to identify individuals and detect mouth movements (speaking indicators).
• Employed Time Difference of Arrival (TDOA) with a microphone array to determine the sound source angle.
• Combined YOLO-derived spatial coordinates with TDOA angles for accurate speaker positioning using inverse trigonometry.

Main Results:

• YOLOX-Tiny achieved a positioning recall rate of 85.2%; TDOA alone reached 88%.
• The integrated YOLOX-Tiny and TDOA method yielded a recall rate of 86.7%, precision of 100%, and accuracy of 94.5%.
• The combined approach demonstrated superior performance compared to using either visual or auditory localization independently.

Conclusions:

• The proposed method effectively locates the active speaker in video conference settings.
• Integration of YOLO object detection and TDOA significantly enhances speaker localization accuracy and reliability.
• This technology offers a robust solution for improving the interactivity and user experience in virtual meetings.