Interactions of auditory and visual stimuli in space and time

The nervous system has evolved to transduce different types of environmental energy independently, for example light energy is transduced by the retina whereas sound energy is transduced by the cochlea. However, the neural processing of this energy is necessarily combined, resulting in a unified percept of a real-world object or event. These percepts can be modified in the laboratory, resulting in illusions that can be used to probe how multisensory integration occurs. This paper reviews studies that have utilized such illusory percepts in order to better understand the integration of auditory and visual signals in primates. Results from human psychophysical experiments where visual stimuli alter the perception of acoustic space (the ventriloquism effect) are discussed, as are experiments probing the underlying cortical mechanisms of this integration. Similar psychophysical experiments where auditory stimuli alter the perception of visual temporal processing are also described.