Development and Evaluation of a Realistic Audio-Visual Multi-Talker Speech Comprehension Test

Current speech comprehension assessments do not provide a reliable measure of listeners' real-life speech comprehension ability, as they fail to capture the complexity and dynamic nature of real-world communicative scenarios. To better align laboratory assessment with real-world listening demands, we developed a new speech comprehension test using audio-visual recordings of spontaneous conversations between two talkers. The conversations were combined with babble noise at three different signal-to-noise ratios (SNRs) to vary task difficulty (Quiet, 0dB, and -5dB). Speech comprehension was assessed using a word monitoring paradigm, in which listeners were given a word category and responded to target words related to that category by pressing a hand-held button. Outcome measures included self-rated listening effort, hit rate, response time (RT), and head turn behavior. Thirty adults with normal hearing were tested. With decreasing SNR, self-reported listening effort increased, hit rate decreased, and RT increased. Participants predominantly oriented toward the active talker, with visual localization achieved through a combination of head and eye movements. Dynamic analyses around conversational floor transfers revealed that, listeners oriented earlier and more consistently toward the upcoming talker in more challenging conditions. None of the behavioral measures were significantly associated with listening effort ratings after accounting for SNR effects. Overall, our results demonstrate that the proposed test provides a sensitive measure of real-world speech comprehension and holds promise for assessing the impact of hearing loss and evaluating hearing device benefit in ecologically valid settings.