Validating a Method to Assess Lipreading, Audiovisual Gain, and Integration During Speech Reception With...

Area of Science:

• Audiology
• Speech Science
• Human-Computer Interaction

Background:

• Watching a speaker's mouth improves speech reception, especially in noisy environments or for individuals with hearing impairments.
• Standardized methods for assessing audiovisual speech reception, including lipreading, are lacking for many languages.
• Audiovisual speech reception measures are valuable for diagnosing and treating hearing disorders.

Purpose of the Study:

• To validate a new method for assessing audiovisual speech reception using a synthetic avatar for standardized lip-sync.
• To evaluate if effects observed with live or filmed talkers in audiovisual speech reception studies can be replicated with synthetic articulation.
• To test hypotheses regarding cochlear implant users' lipreading abilities, age-related differences, and the benefits of audiovisual speech over unimodal speech.

Main Methods:

• A cross-sectional study involving 14 experienced cochlear implant (CI) users and 14 normal-hearing (NH) individuals.
• Assessment of lipreading, audiovisual gain, and integration using a German matrix sentence test.
• Utilization of the 'MASSY' (Modular Audiovisual Speech Synthesizer) system to generate standardized visual speech stimuli from a synthetic avatar.

Main Results:

• CI users demonstrated significantly higher visual-only speech reception (38%) compared to NH individuals (12%).
• Younger participants in both groups achieved higher lipreading scores, indicating an age-related effect.
• Both CI and NH groups benefited from audiovisual speech presentation over unimodal auditory or visual input, showing significant visual enhancement, auditory enhancement, and integration enhancement.

Conclusions:

• The validated method, supplementing auditory tests with synthetic visual cues, offers a standardized approach for assessing audiovisual speech reception.
• This method is suitable for clinical and scientific applications, aiding in the diagnosis and treatment of hearing and communication disorders.
• The findings support the utility of synthetic avatars in creating controlled and reproducible audiovisual speech reception assessments.