Leveraging Deep Learning to Enhance Optical Microphone System Performance with Unknown Speakers for Cochlear Implants

Cochlear implants (CI) play a crucial role in restoring hearing for individuals with profound-to-severe hearing loss. However, challenges persist, particularly in low signal-to-noise ratios and distant talk scenarios. This study introduces an innovative solution by integrating a Laser Doppler vibrometer (LDV) with deep learning to reconstruct clean speech from unknown speakers in noisy conditions. Objective evaluations, including short-time objective intelligibility (STOI) and perceptual evaluation of speech quality (PESQ), demonstrate the superior performance of the proposed-LDV system over traditional microphones and a baseline LDV system under the same recording conditions. STOI scores for Mic-Noisy, Mic-log Minimum Mean Square Error (logMMSE), baseline-LDV, and proposed-LDV were 0.44, 0.35, 0.48, and 0.73, respectively, whereas PESQ scores were 1.51, 1.76, 1.4, 0.73, and 1.96, respectively. Furthermore, the vocoder simulation listening testing results showed the proposed system achieving a higher word accuracy score than baselines systems. These findings highlight the potential of the proposed system as a robust speech capture method for CI users, addressing challenges related to noise and distance.