Temporally congruent auditory stream modulates visual processing both independently of and interactively with...

In competing environments, both selective attention and audiovisual interaction can facilitate visual processing, yet whether their influences operate independently or interactively remains debated. Using electroencephalography (EEG), we addressed this issue by instructing participants to selectively attend to one of two lateralized flickering discs, which also changed their shapes either temporally congruent or incongruent with a pitch-changing sound. We found that reaction times for detecting deviants embedded in the attended visual stream were reduced when a temporally congruent sound was concurrently played. Compared to a temporally incongruent auditory stream, a congruent one selectively enhanced steady-state visual evoked potentials (SSVEPs) in response to flickering of the unattended stream. In contrast, the SSVEP and inter-trial phase coherence in response to the shape-modulation for both attended and unattended streams were enhanced at the harmonic frequencies by the temporally congruent sound. The results indicate that the auditory influence on visual processing orthogonal to audiovisual temporal congruency (flicker) interacts with attention, whereas the auditory influence on visual processing relevant to audiovisual temporal congruency (shape-modulation) is largely independent of attention. However, these congruency effects were observed only under rhythmic audiovisual streams: When audiovisual pitch-shape modulation followed unrhythmic temporal structures, these congruency effects totally disappeared. Together, these findings demonstrate that temporally congruent auditory streams can modulate visual processing both independently of and interactively with selective attention, highlighting a flexible and complex interplay between selective attention and audiovisual interaction.