Acoustic space learning for sound-source separation and localization on binaural manifolds

Area of Science:

• Acoustics
• Signal Processing
• Machine Learning

Background:

• Modeling acoustic spaces and localizing/separating multiple sound sources are complex challenges.
• Existing methods struggle with high-dimensional spectral data and simultaneous sparse-spectrum sounds.

Purpose of the Study:

• To introduce the binaural manifold paradigm for acoustic space modeling.
• To develop a robust method for 2D sound source localization and separation.
• To handle high-dimensional interaural spectral data and real-world spectrogram imperfections.

Main Methods:

• Utilized nonlinear dimensionality reduction to reveal a 2D manifold in interaural spectral data.
• Proposed a probabilistic piecewise affine mapping model (PPAM) for high-dimensional data.
• Developed a variational Expectation-Maximization (EM) framework (VESSL) for multi-source scenarios.

Main Results:

• Demonstrated that acoustic data lies on a 2D manifold parameterized by listener/source direction.
• Achieved accurate 2D localization and separation of multiple sound sources, outperforming state-of-the-art methods.
• Successfully handled missing data and redundancy in spectrograms for natural sound source localization.

Conclusions:

• The binaural manifold paradigm combined with Bayesian inference offers superior performance in acoustic source localization and separation.
• The proposed PPAM and VESSL algorithms provide a powerful framework for complex auditory scene analysis.
• This research advances the understanding and computational modeling of binaural hearing.