Related Experiment Video
Updated: Jul 18, 2026

Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
The perception of visual images encoded in musical form: a study in cross-modality information transfer
J Cronly-Dillon1, K Persaud, R P Gregory
1Department of Optometry and Neuroscience, UMIST, UK. mjcjcd@fsl.op.umist.ac.uk
Abstract:
This study demonstrates the ability of blind (previously sighted) and blindfolded (sighted) subjects in reconstructing and identifying a number of visual targets transformed into equivalent musical representations. Visual images are deconstructed through a process which selectively segregates different features of the image into separate packages. These are then encoded in sound and presented as a polyphonic musical melody which resembles a Baroque fugue with many voices, allowing subjects to analyse the component voices selectively in combination, or separately in sequence, in a manner which allows a subject to patch together and bind the different features of the object mentally into a mental percept of a single recognizable entity. The visual targets used in this study included a variety of geometrical figures, simple high-contrast line drawings of man-made objects, natural and urban scenes, etc., translated into sound and presented to the subject in polyphonic musical form.
Related Concept Videos
Perception of Sound Waves
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
Sensory Modalities
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
Perception
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Auditory Perception
Factors Affecting Perception
An illustrative example of a perceptual set is the scenario where an airline pilot told...
Synesthesia

