Optical Microphone-Based Speech Reconstruction System With Deep Learning for Individuals With Hearing Loss

Area of Science:

• Biomedical Engineering
• Signal Processing
• Hearing Science

Background:

• Conventional speech enhancement (SE) methods struggle with non-stationary noise and distant speakers.
• Existing SE algorithms often fail to adequately improve speech perception for hearing-impaired patients in challenging acoustic environments.

Purpose of the Study:

• To overcome the limitations of conventional speech enhancement approaches.
• To develop a novel SE method capable of enhancing speech quality and intelligibility for hearing-impaired individuals.

Main Methods:

• A speaker-closed, deep learning-based SE method was proposed.
• An optical microphone was integrated to capture target speaker audio.
• The method was evaluated for seven typical hearing loss types.

Main Results:

• The proposed method significantly outperformed baseline methods in objective evaluations.
• Improvements of 0.21-0.27 in speech quality (HASQI) and 0.34-0.64 in speech comprehension/intelligibility (HASPI) were observed.
• The method effectively reduced noise and mitigated distance-related interference.

Conclusions:

• The proposed deep learning-based SE method enhances speech perception by filtering noise and compensating for distance.
• This approach offers a promising solution to improve the listening experience for individuals with hearing impairments.