Audio-visual consonant recognition with the 3M/House cochlear implant

Area of Science:

• Audiology
• Speech Science
• Biomedical Engineering

Background:

• Cochlear implants restore partial hearing but speech perception remains a challenge.
• Understanding how different sensory modalities contribute to speech recognition is crucial for improving implant technology.

Purpose of the Study:

• To evaluate consonant recognition in cochlear implant users across auditory-only, visual-only, and combined auditory-visual conditions.
• To analyze the transmission of phonetic features in each sensory condition.

Main Methods:

• Eight experienced 3M/House cochlear implant users participated.
• Consonant recognition was tested using videotaped vowel-consonant-vowel lists.
• Stimuli were presented in auditory-only (A), visual (V), and auditory-visual (AV) conditions.

Main Results:

• All subjects performed significantly better than chance across all conditions.
• Auditory-visual (AV) scores were significantly higher than visual (V) scores, which were higher than auditory-only (A) scores.
• Sequential Information Analysis revealed distinct feature transmission patterns: sonorant and voicing features in A; place features in V; both place and acoustic features in AV, with visual cues having the most influence.

Conclusions:

• Combined auditory-visual input significantly enhances consonant recognition in cochlear implant users compared to unimodal input.
• Different phonetic features are better transmitted through auditory or visual information, with combined input leveraging the strengths of both.
• Visual cues play a critical role in transmitting place and acoustic features in the auditory-visual condition.