Related Experiment Video
Updated: Aug 26, 2025

Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
Cross-modal associations between paintings and sounds: Effects of embodiment
Marina Iosifyan1, Anton Sidoroff-Dorso2, Judith Wolfe1
17486University of St Andrews, School of Divinity, UK.
Abstract:
The present study investigated cross-modal associations between a series of paintings and sounds. We studied the effects of sound congruency (congruent vs. non-congruent sounds) and embodiment (embodied vs. synthetic sounds) on the evaluation of abstract and figurative paintings. Participants evaluated figurative and abstract paintings paired with congruent and non-congruent embodied and synthetic sounds. They also evaluated the perceived meaningfulness of the paintings, aesthetic value and immersive experience of the paintings. Embodied sounds (sounds associated with bodily sensations, bodily movements and touch) were more strongly associated with figurative paintings, while synthetic sounds (non-embodied sounds) were more strongly associated with abstract paintings. Sound congruency increased the perceived meaningfulness, immersive experience and aesthetic value of paintings. Sound embodiment increased immersive experience of paintings.
Related Concept Videos
Perception of Sound Waves
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
Synesthesia
Auditory Perception
Sensory Modalities
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
Perceiving Loudness, Pitch, and Location
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Perception
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...

