Where on the face do we look during phonemic restoration: An eye-tracking study

Area of Science:

• Speech perception
• Auditory-visual speech processing
• Human-computer interaction

Background:

• Face-to-face communication integrates auditory and visual speech cues.
• Understanding how task demands influence visual attention during speech processing is crucial for real-world scenarios.

Purpose of the Study:

• To investigate the impact of task demands on gaze patterns towards a speaking face.
• To determine how visual articulatory information affects speech perception under varying cognitive loads.

Main Methods:

• Two eye-tracking experiments were conducted using audiovisual and pixelated visual conditions.
• Participants engaged in passive (no response) or active (speech discrimination) tasks.
• Speech stimuli included clear /ba/ and modified /a/-like tokens.

Main Results:

• Maximal fixation to the mouth occurred in the audiovisual active condition.
• Visual articulatory information facilitated a phonemic restoration effect for the /a/-like token.
• In the pixelated condition, participants fixated on the eyes; active discrimination was better than in the audiovisual condition.

Conclusions:

• Adults utilize visual articulatory cues from the mouth to disambiguate speech when available and task demands are high.
• Gaze patterns shift towards the mouth or eyes depending on visual information availability and task requirements.
• Visual speech information plays a significant role in supporting auditory speech processing, particularly in challenging listening environments.