Evaluating the Performance of a Visually Guided Hearing Aid Using a Dynamic Auditory-Visual Word Congruence Task

Area of Science:

• Auditory Neuroscience
• Assistive Listening Devices
• Human-Computer Interaction

Background:

• Beamforming technology enhances speech identification in fixed sound environments.
• Visually guided hearing aids (VGHAs) use eye gaze to steer beamforming microphones.
• Previous research shows benefits of non-visually steered beamforming.

Purpose of the Study:

• To evaluate the effectiveness of a simulated VGHA in dynamic listening conditions.
• To assess VGHA performance when target speech locations change unpredictably.
• To compare VGHA performance with natural binaural listening and other conditions.

Main Methods:

• An auditory-visual word congruence task was used with three simultaneous speech sources.
• Participants included young normal-hearing (NH) and hearing-impaired (HI) adults.
• Performance was compared across natural binaural, simulated VGHA (BEAM), and hybrid (BEAMAR) conditions, with fixed and dynamic target locations.

Main Results:

• Significant benefits were observed for the hybrid BEAMAR condition in fixed target scenarios for both NH and HI groups.
• Individual performance varied, with some showing direct BEAM benefits.
• Under dynamic conditions, performance dipped after target transitions but recovered rapidly.

Conclusions:

• The benefits of beamforming are largely maintained in dynamic conditions with eye-gaze steering.
• The VGHA demonstrates rapid performance recovery following unpredictable sound source transitions.
• The findings support the potential of VGHAs for improving speech perception in complex auditory environments for both NH and HI users.