What You See Is What You Hear: Sounds Alter the Contents of Visual Perception

Area of Science:

• Cognitive Psychology
• Neuroscience
• Sensory Integration

Background:

• Visual object recognition relies on context and prior knowledge.
• The role of auditory context in visual perception is not fully understood.

Purpose of the Study:

• To investigate the influence of auditory context on visual object perception.
• To determine if auditory information can shape the representation of visual objects.

Main Methods:

• A psychophysical task pairing naturalistic sounds with noisy visual inputs.
• Experiments with young adults to assess visual object recognition under different auditory conditions.
• Control experiments to rule out decision biases and top-down expectations.

Main Results:

• Ambiguous visual object representations shifted towards features suggested by incidental sounds.
• Effects were not caused by response biases or top-down expectations.
• Continuous integration of audiovisual inputs drove the observed perceptual shifts.

Conclusions:

• Auditory context critically shapes visual object perception.
• Naturalistic sounds provide independent and diagnostic information that directly influences visual experience.
• Perceptual experience is a dynamic integration of multisensory information.