3D joint speaker position and orientation tracking with particle filters

Area of Science:

• Acoustics and Signal Processing
• Human-Computer Interaction
• Robotics and Artificial Intelligence

Background:

• Accurate speaker localization and orientation are crucial for intelligent environments.
• Existing methods often treat localization and orientation estimation separately, limiting performance.
• Smart rooms require robust systems for understanding user position and focus.

Purpose of the Study:

• To develop a joint Bayesian approach for simultaneous 3D speaker location and orientation estimation.
• To leverage the interdependency between speaker location and orientation for improved accuracy.
• To validate the proposed method using a novel multimodal dataset.

Main Methods:

• A Bayesian framework is employed for joint tracking of speaker location and orientation.
• Speaker mouth location is estimated, and head center is deduced from orientation.
• Elevation angles are inferred from vertical mouth movements.
• An Inertial Measurement Unit (IMU) dataset provides ground truth 3D orientation.

Main Results:

• The proposed joint algorithm demonstrates superior performance over a two-step approach.
• Enhanced precision in both speaker localization and orientation angle estimation was achieved.
• The joint estimation significantly improved the accuracy of 3D speaker tracking.

Conclusions:

• Jointly estimating speaker location and orientation provides significant performance benefits.
• The Bayesian approach offers a robust solution for 3D speaker analysis in smart rooms.
• This work advances the capabilities of intelligent environments through improved human-centric sensing.