A versatile deep-neural-network-based music preprocessing and remixing scheme for cochlear implant listeners.

Area of Science:

• Auditory Neuroscience
• Signal Processing
• Machine Learning

Background:

• Cochlear implants (CIs) significantly restore speech perception but offer limited music perception for users.
• Existing CI technology struggles to convey the complex spectral and temporal information crucial for music enjoyment.

Purpose of the Study:

• To develop and evaluate a deep learning-based signal preprocessing strategy to simplify music signals for CI users.
• To emphasize rhythmic information in music to improve its accessibility for individuals with hearing impairments.

Main Methods:

• Proposed a methodology combining harmonic/percussive source separation with deep neural network (DNN) based source separation.
• Utilized a versatile source mixture model and assessed two different DNN architectures.
• Evaluated the method using instrumental measures and listening experiments with normal-hearing subjects, including vocoded conditions approximating CI listening.

Main Results:

• Four combinations of remix models and DNNs were evaluated with vocoded signals.
• All tested combinations significantly improved perceived signal quality compared to unprocessed signals.
• The two best-performing remix networks demonstrated promising results for potential use with actual CI listeners.

Conclusions:

• The proposed deep learning-based signal preprocessing effectively simplifies music and enhances rhythmic information for improved perception.
• The developed method shows significant potential for enhancing the music listening experience of cochlear implant users.
• Further evaluation with CI listeners is warranted for the most promising network and remix model combinations.