Related Experiment Video
Updated: Feb 26, 2026
13:51
Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
20.6K
Summation of visual attributes in auditory-visual crossmodal correspondences
Clare Jonas1, Mary Jane Spiller2, Paul Hibbard3
1School of Psychology, University of East London, London, UK. c.n.jonas@uel.ac.uk.
Psychonomic Bulletin & Review
|July 12, 2017
Summary
Human perception links senses, like sound pitch to object size. This study found visual features generally combine to influence auditory-visual matching, with implications for sensory substitution technology.
Area of Science:
- Cognitive Psychology
- Neuroscience
- Human-Computer Interaction
Background:
- Crossmodal correspondences link different sensory dimensions (e.g., sound pitch and object size).
- Previous research has not systematically explored interactions between visual and auditory crossmodal correspondences.
Purpose of the Study:
- To investigate how visual properties (luminance, saturation, size, vertical position) influence auditory-visual matching.
- To examine the combined effects of multiple visual dimensions on crossmodal perception.
Main Methods:
- Participants matched visual stimuli with high-pitched or low-pitched auditory stimuli.
- Visual stimuli varied systematically in luminance, saturation, size, and vertical position.
- Analysis focused on how these visual dimensions interacted to affect matching decisions.
Main Results:
- A general pattern of summation was observed, where individual crossmodal correspondences combined predictably.
- Some exceptions to summation were noted, potentially explained by Garner interference.
- Visual dimensions significantly influenced the matching of auditory pitches to visual stimuli.
Conclusions:
- Visual features generally summate in influencing crossmodal perception between vision and audition.
- Understanding these interactions is crucial for designing effective sensory substitution systems.
- Findings contribute to the broader understanding of human sensory integration and perception.
Related Concept Videos
01:26Visual System
2.1K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
2.1K
01:27Synesthesia
777
Synesthesia is a remarkable condition where stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. People with synesthesia experience a blending or crossing of their senses, such as sight and sound, leading to cross-modal sensations. In this condition, the stimulation of one sense, such as hearing a number or musical note, triggers an experience of another sense, like sensing a specific color, taste, or smell. People...
777
01:15Auditory Pathway
7.7K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
7.7K
01:21Association Areas of the Cortex
9.9K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
9.9K
01:14Motor and Sensory Areas of the Cortex
8.3K
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
8.3K
01:24Vision
60.7K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
60.7K