Related Experiment Video
Updated: Sep 13, 2025
09:09
Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
528
Robust frame-level speaker localization guided by multi-channel speech enhancement and inter-channel phase-difference
Shanmukha Srinivas Battula1, Hassan Taherian1, Ashutosh Pandey2
1Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio 43210, USA.
The Journal of the Acoustical Society of America
|July 28, 2025
Summary
This study enhances speaker localization in noisy rooms using complex spectral mapping (CSM) for speech enhancement and direction-of-arrival (DOA) estimation. Multi-input multi-output (MIMO) systems with new loss functions significantly improve accuracy over single-output systems.
Area of Science:
- Signal Processing
- Acoustics
- Machine Learning
Background:
- Frame-level speaker localization is challenging in reverberant and noisy environments.
- Existing deep learning methods often process multi-channel audio directly, limiting performance.
- Accurate sound source localization relies on preserving inter-channel phase information.
Purpose of the Study:
- To develop a robust frame-level speaker localization method for adverse acoustic conditions.
- To investigate multi-channel speech enhancement using complex spectral mapping (CSM) for improved direction-of-arrival (DOA) estimation.
- To propose and evaluate novel multi-input multi-output (MIMO) systems and loss functions for enhanced localization accuracy.
Main Methods:
- Multi-channel speech enhancement using complex spectral mapping (CSM).
- Direction-of-arrival (DOA) estimation via weighted generalized cross-correlation with phase transform (GCC-PHAT).
- Comparison of multi-input single-output (MISO) and multi-input multi-output (MIMO) models, including new loss functions incorporating phase differences.
Main Results:
- CSM-derived phase estimates are reliable for frame-level DOA estimation.
- MIMO systems demonstrate superior performance compared to MISO systems.
- The proposed model achieves excellent frame-level speaker localization, outperforming related methods significantly.
Conclusions:
- The proposed CSM-based speech enhancement and DOA estimation framework effectively addresses limitations in noisy and reverberant environments.
- MIMO systems with novel phase-aware loss functions offer substantial improvements in speaker localization accuracy.
- The method achieves state-of-the-art frame-level localization performance, even surpassing utterance-level results of other approaches.
Related Concept Videos
01:10Interference: Path Lengths
1.4K
Consider two sources of sound, that may or may not be in phase, emitting waves at a single frequency, and consider the frequencies to be the same.
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...
1.4K
01:21Perceiving Loudness, Pitch, and Location
429
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
429
01:26Linear Approximation in Frequency Domain
135
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
135
01:10Reconstruction of Signal using Interpolation
345
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
345
01:15Distance Corrections
83
To achieve precise distance measurements, especially in surveying and construction, certain corrections must be applied to account for potential sources of error like the standardization errors, temperature variations, and slope adjustments.Standardization error emerges when measurement equipment undergoes changes, such as wear, repairs, or weather impacts. To address this, surveyors compare the equipment’s readings to a standard. This process identifies any deviation that might lead to...
83
01:08IR Spectrum Peak Splitting: Symmetric vs Asymmetric Vibrations
1.2K
Identical bonds within a polyatomic group can stretch symmetrically (in-phase) or asymmetrically (out-of-phase). Similar to hydrogen bonding, these vibrations also influence the shape of the IR peak. Generally, asymmetric stretching frequencies are higher than symmetric stretching frequencies. For example, primary amines exhibit two distinct IR peaks between 3300–3500 cm−1 corresponding to the symmetric and asymmetric N-H stretching, while secondary amines exhibit a single...
1.2K