Semantically congruent audiovisual integration with modal-based attention accelerates auditory short-term memory...

Area of Science:

• Cognitive Neuroscience
• Psychology
• Sensory Perception

Background:

• Multisensory integration offers asymmetric benefits to unisensory perception.
• Auditory perception often shows greater benefits, particularly with attention on irrelevant visual stimuli.
• The impact of semantic congruence and attention on multisensory integration for short-term memory (STM) retrieval is not well understood.

Purpose of the Study:

• To investigate whether multisensory integration benefits for subsequent unisensory STM retrieval are asymmetric.
• To examine the role of semantic congruence and modal-based attention during multisensory memory encoding.

Main Methods:

• Employed a delayed matching-to-sample paradigm.
• Manipulated attention focus during multisensory memory encoding.
• Compared unisensory and semantically congruent/incongruent multisensory encoding conditions.

Main Results:

• Both visual and auditory STM retrieval were faster following semantically congruent multisensory encoding compared to unisensory encoding.
• Auditory STM retrieval showed a specific acceleration solely due to semantically congruent multisensory encoding.
• This suggests that attention optimizes multisensory representation formation for memory retrieval.

Conclusions:

• Coherent multisensory representation formation, optimized by attention, can be rapidly triggered during unisensory memory retrieval.
• Auditory STM retrieval benefits significantly from prior congruent multisensory experiences, indicating a reliance on robust integrated representations.
• The findings underscore the asymmetric nature of multisensory benefits in memory retrieval, particularly for less dominant sensory modalities.