Psychoacoustically inspired optimization framework for neural binaural synthesis

Area of Science:

• Audio Signal Processing
• Computational Auditory Perception
• Machine Learning for Audio

Background:

• Current neural binaural synthesis methods use signal-oriented objectives, neglecting human auditory perception.
• Uniform treatment of time-frequency regions can lead to artifacts, especially in reverberant sound fields.
• Perceptually masked or sub-threshold components are often overemphasized.

Purpose of the Study:

• To develop a perceptual optimization framework for neural binaural synthesis.
• To improve spatial accuracy and signal fidelity in immersive audio reproduction.
• To address limitations of signal-oriented optimization in current methods.

Main Methods:

• Introduced a coherence-gated spatial loss using interaural coherence to weight spatial cues.
• Reduced contribution of low-coherence regions to prioritize direct sound and early reflections.
• Incorporated a multi-scale wavelet loss for error measurement across decomposition levels.

Main Results:

• The framework demonstrated consistent improvements over state-of-the-art baselines.
• Objective evaluations and subjective listening tests confirmed enhanced performance.
• Validated on three distinct backbone architectures, showing model-agnostic effectiveness.

Conclusions:

• The proposed perceptual framework significantly enhances neural binaural synthesis.
• It effectively prioritizes perceptually relevant audio components for better localization.
• This approach offers a robust training strategy for immersive audio reproduction.