Semantic incongruity influences response caution in audio-visual integration.

Area of Science:

• Cognitive Neuroscience
• Psychology
• Sensory Integration

Background:

• Multisensory stimulus combinations typically yield shorter reaction times (RTs) than single-modality stimuli, a phenomenon known as inter-sensory facilitation.
• This facilitation has been hypothesized to occur only with semantically congruent stimuli, as incongruity was thought to hinder multisensory integration.

Purpose of the Study:

• To investigate whether semantic incongruity prevents multisensory integration or influences response caution.
• To examine the role of response caution in the observed effects of congruency on reaction times.

Main Methods:

• Two experiments utilized two-alternative forced-choice decision tasks with auditory, visual, and audio-visual stimuli.
• Stimuli pairs were either semantically congruent (e.g., ambulance image and horn sound) or incongruent (e.g., ambulance image and bell sound).
• Bayesian hierarchical drift diffusion modeling was employed to analyze response times and decision-making processes.

Main Results:

• Shorter RTs and race model inequality violations were observed for congruent stimuli, consistent with prior research.
• Bayesian analyses indicated that longer RTs for incongruent stimuli result from an 'incongruity caution effect,' not a failure of integration.
• Response caution was found to be dynamically adjusted on a trial-by-trial basis in response to incoming sensory information.

Conclusions:

• The effect of semantic incongruity on reaction times is primarily mediated by increased response caution, rather than a breakdown of multisensory integration.
• Cognitive control mechanisms allow for adaptive adjustments in response caution based on the semantic consistency of incoming multisensory information.
• Stimulus modality also impacts non-cognitive aspects of response generation, suggesting complex interactions in multisensory processing.