3 directional Inception-ResUNet: Deep spatial feature learning for multichannel singing voice separation with...

Area of Science:

• Robotics
• Signal Processing
• Artificial Intelligence

Background:

• Humanoid robots face challenges in interpreting complex auditory signals, such as mixed singing voices, music, and noise.
• Acoustic signals perceived by robots are often distorted, attenuated, and reverberated, complicating voice separation tasks.

Purpose of the Study:

• To develop an advanced singing voice separation model for humanoid robots.
• To enhance the robot's ability to interpret ambiguous auditory signals in noisy environments.

Main Methods:

• Utilized a 3D Inception-ResUNet architecture within a U-shaped network to process spectrograms.
• Employed multi-objective training with magnitude consistency loss, phase consistency loss, and magnitude correlation consistency loss.
• Synthesized a 10-channel dataset using NAO robots and the MIR-1K dataset for model training.

Main Results:

• The proposed model achieved an average Normalized Source-to-Distortion Ratio (NSDR) of 11.55 dB on the test dataset.
• Demonstrated superior performance compared to existing comparison models in singing voice separation tasks.
• The multi-objective training strategy effectively improved the utilization of spatial and spectral information.

Conclusions:

• The 3D Inception-ResUNet model offers a significant advancement in robot-based singing voice separation.
• Multi-objective training is crucial for enhancing the accuracy and robustness of auditory signal interpretation in robots.
• This research contributes to more sophisticated human-robot interaction through improved audio processing capabilities.