A Real-Time Convolutional Neural Network Based Speech Enhancement for Hearing Impaired Listeners Using Smartphone

Area of Science:

• Signal Processing
• Artificial Intelligence
• Assistive Technology

Background:

• Hearing aid users often experience degraded speech quality and intelligibility in noisy environments.
• Existing speech enhancement (SE) techniques may not be suitable for real-time, on-device applications.

Purpose of the Study:

• To develop and evaluate a real-time SE technique using a convolutional neural network (CNN) for hearing aid (HA) users.
• To implement the SE technique as a smartphone application for seamless integration with HAs.

Main Methods:

• A multi-objective learning CNN model was employed for noise removal from noisy speech spectra.
• The SE algorithm was optimized for computational efficiency and low processing delay for smartphone implementation.
• The developed application was tested on a smartphone processor, functioning as an assistive tool for HAs.

Main Results:

• The proposed CNN-based SE technique demonstrated significant improvements in speech quality and intelligibility metrics compared to conventional and other neural network-based methods.
• The real-time SE application showed seamless operation on a smartphone processor.
• Experimental results confirmed the usability of the SE application in various noisy conditions.

Conclusions:

• The developed smartphone-based SE application effectively enhances speech for hearing aid users.
• The multi-objective learning CNN model offers a computationally efficient and low-delay solution for real-time SE on mobile devices.
• This work presents a viable assistive technology for improving auditory perception in noisy environments.