Loudness changes as a function of the audiovisual scene

Understanding how listeners perceive speech in complex audiovisual (AV) environments is essential for characterizing communication in everyday settings. Here, we used loudness ratings to examine how AV scene characteristics shape the perceptual representation of a target talker in multitalker babble. Twenty-five normal-hearing adults rated the perceived loudness of a female talker presented in four-talker babble at four target-to-masker ratios (TMRs: - 21, - 9, - 6, - 3 dB). Audiovisual temporal coherence was manipulated by delaying the audio relative to the video at five stimulus onset asynchronies (SOAs: 0-500 ms), and three linguistic categories were tested: words, pseudowords, and reversed words. At synchronous presentations, loudness ratings scaled with TMR and were higher for words than pseudowords and reversed words, despite identical physical intensities. Increasing AV asynchrony reduced perceived loudness, with significant drops emerging beyond 150-300 ms SOA. Mapping loudness ratings onto perceived TMR change using individual loudness-growth slopes eliminated the main effect of the linguistic category, isolating a 2-3 dB perceived TMR drop at 500 ms SOA across all categories. These results demonstrate that both AV temporal synchrony and linguistic content shape loudness perception in noise, but their contributions are dissociable. This provides a novel view into how the perceptual construct of loudness changes with the AV scene, with implications for how loudness ratings can be used to study scene analysis and communication.