Task irrelevant sounds influence visual attention through graded crossmodal semantic modulation

Area of Science:

• Cognitive Psychology
• Neuroscience
• Multisensory Processing

Background:

• Visual attention is guided by semantic information.
• Less is known about how semantics shape attention in multisensory contexts.
• Task-irrelevant sounds can facilitate visual search for matched targets.

Purpose of the Study:

• To investigate if semantic information influences attention in audiovisual contexts beyond direct matches.
• To determine if sound-image semantic relatedness affects visual search speed.
• To examine if this effect persists when semantic information is task-irrelevant.

Main Methods:

• Participants searched for target images while semantic relatedness between sounds and images was systematically varied.
• Two follow-up experiments used an orthogonal Gabor discrimination task to assess task relevance.
• Behavioral search speed was the primary measure.

Main Results:

• Visual search speed scaled with sound-image semantic relatedness; targets were found faster with more related sounds.
• This semantic scaling effect persisted even when the sounds were task-irrelevant.
• Semantic knowledge guides audiovisual attention robustly, irrespective of task relevance or direct matching.

Conclusions:

• Semantic knowledge significantly guides audiovisual attention.
• A robust cognitive mechanism processes semantics across sensory modalities to influence attentional allocation.
• This mechanism operates even for task-irrelevant stimuli and beyond direct sensory matches.