Speech-in-Noise Ability and Signal to Noise Ratio Predict the Timing of Hearing-Impaired Listeners' Intertalker Saccades When Observing Conversational Turn-Taking: An Explorative Investigation

We explored the hypothesis that, when listeners visually follow the turn-taking of talkers engaged in a conversation, the timing of their eye movements is related to their ability to follow the conversation.

We made use of a re-purposed dataset where adults with hearing impairment (N = 17), assisted by hearing aids, observed audiovisual recordings of dyadic conversations presented via a television screen and loudspeakers. The recordings were presented with multitalker babble noise at four signal to noise ratios (SNRs), in 4-dB steps ranging from -4 to 8 dB, to modulate the participants' ability to follow the conversation. We extracted time windows around conversation floor transfers (FTs) in the stimulus where participants reacted by moving their gaze from one talker to the next, termed FT-intertalker saccades (ITS). We recorded the timing of this eye movement relative to the onset of the new talker's speech. In addition, participants completed a separate word-recognition test to measure their speech perception in noise (SPIN) ability at the same SNRs as used for the conversation stimuli. We predicted that the timing of FT-ITS would be delayed with difficult SNR levels and for listeners with low SPIN ability. The effect of SPIN ability was tested first as a continuous variable, and subsequently with participants divided into high and low SPIN-ability groups.

Multilevel linear modeling showed that the timing of FT-ITS was predicted by SNR condition and SPIN group, but no effect was found for SPIN ability as a continuous variable. Post hoc comparisons (uncorrected for multiple comparisons) indicated that delayed FT-ITS were associated with low SPIN ability, and both the hardest and easiest SNR conditions. The full model accounted for 34.5% of the variance in the data, but the fixed effects of SPIN and SNR together accounted for only 2.3%.

Although the results should be interpreted with caution due to limitations in the experiment design, they provide preliminary support that FT-ITS timing can be used as a measure of hearing-impaired listeners' ability to follow a conversation. This first exploration of this question can serve future studies on this topic, providing guidance on the range of perceptual difficulty where this measure may be sensitive, and recommending a modeling approach that takes into account differences between stimuli.