The modality-switch effect: visually and aurally presented prime sentences activate our senses

Area of Science:

• Cognitive Psychology
• Neuroscience
• Linguistics

Background:

• The modality-switch effect describes the cognitive cost incurred when verifying concept properties across different sensory modalities.
• This effect is hypothesized to stem from perceptual simulation processes.
• Existing research suggests this cost may involve the activation of sensory information.

Purpose of the Study:

• To investigate if the modality-switch effect is caused by the activation of sensory information.
• To determine if linguistically described stimuli can trigger this sensory activation.
• To explore the role of presentation modality (written vs. aural) in the modality-switch effect.

Main Methods:

• Participants were primed with sentences describing visual or auditory properties.
• A property-verification task was administered on target sentences.
• Stimuli were presented in either written or aural formats, manipulating modality compatibility between prime and target.

Main Results:

• A significant processing cost was observed when participants switched between different sensory modalities (e.g., visual to auditory).
• This cost was evident regardless of whether stimuli were presented visually or aurally.
• Repeating the same modality led to lower processing costs compared to switching modalities.

Conclusions:

• The findings support the embodied and grounded cognition view, suggesting conceptual knowledge is rooted in perceptual systems.
• Linguistic stimuli, whether read or listened to, can pre-activate specific sensory modalities.
• This pre-activation contributes to the modality-switch effect, demonstrating a link between language processing and sensory experience.