Effects of multisensory simultaneity judgment training on the comprehension and cortical processing of speech in...

Understanding speech in noise can be facilitated by integrating auditory and visual speech cues. Audiovisual temporal acuity, which can be indexed by the temporal binding window (TBW), is critical for this process and can be enhanced through simultaneity judgment training. We hypothesized that multisensory training would narrow the TBW and improve speech understanding in noise. Participants were randomized to receive either training and testing (n = 15) or testing-only (n = 15) over three days. Trained participants demonstrated significant narrowing in their mean TBW size (403ms to 345ms; p = 0.030), whereas control participants did not (409ms to 474ms; p = 0.061). Although there were no group-level changes in word recognition scores, trained participants with larger TBW decreases exhibited larger improvements in auditory word recognition in noise (R² = 0.291; p = 0.038). Individual differences in responses to training were found to be related to differences in cortical speech processing using functional near-infrared spectroscopy. Low audiovisual-evoked activity in the left middle temporal gyrus (R² = 0.87; p = 0.006), left angular and superior temporal gyrus (R² = 0.85; p = 0.006), and visual cortices (R² = 0.74; p = 0.041) was associated with larger improvements in auditory word recognition after training. Multisensory training transfers benefits to speech comprehension in noise, and this effect may be mediated by upregulating activity in multisensory cortical networks for individuals with low baseline activity.