How actions shape perception: learning action-outcome relations and predicting sensory outcomes promote audio-visual...

Area of Science:

• Neuroscience
• Cognitive Science
• Sensory Integration

Background:

• The brain unifies audio-visual information to create a cohesive perception of objects.
• Recalibration of audio-visual simultaneity adjusts for signal transmission and processing delays.
• Instrumental actions and their perceptual outcomes play a role in sensory integration.

Purpose of the Study:

• To investigate the influence of instrumental action control on audio-visual recalibration.
• To determine if learning action-outcome associations affects the temporal binding of sensory events.
• To explore how prediction of perceptual outcomes contributes to temporal unification.

Main Methods:

• Four experiments were conducted involving participants learning specific audio-visual stimuli linked to distinct hand actions (button-presses).
• Participants experienced either audio preceding visual stimuli or audio lagging visual stimuli based on their action.
• A test phase assessed recalibration by presenting learned or alternative audio-visual outcomes for each action.

Main Results:

• Audio-visual recalibration of simultaneity was observed exclusively for previously learned action-outcome pairs.
• Learning a specific action-outcome relation promoted temporal grouping of associated audio and visual events.
• This temporal grouping contributed to the creation of a unified multisensory object representation.

Conclusions:

• Learning action-outcome relations significantly influences the brain's recalibration of audio-visual simultaneity.
• The prediction of perceptual outcomes aids in establishing an integrated temporal structure for multisensory experiences.
• This process is crucial for creating a temporally unified representation of environmental objects.