Spatio-temporal constraints for auditory--visual integration

The perceptual coherence of auditory and visual information is achieved by integrative brain processes. Specialized single neurons with spatial and temporal interactions of auditory and visual stimuli have been demonstrated by several neurophysiological studies. The present, psychophysical, study investigates possible perceptual correlates of these neuronal features. Subjects had to indicate the point of subjective spatial alignment (PSSA) for a horizontally moving visual stimulus that crossed the position of a stationary sound source. Auditory and visual stimuli consisted of periodic pulses that were systematically varied in their phase relationship or repetition rate. PSSAs obtained for continuous visual stimuli served as a reference. When sound and light pulses were coincident in phase at a repetition rate of 2 Hz, PSSAs were shifted by approximately 3 degrees in a direction opposite to the movement of the visual stimulus (with respect to the reference condition). This shift markedly decreased when the temporal disparity exceeded approximately 100 ms and disappeared near phase opposition (250 ms disparity). With 4 Hz repetition rate (temporal disparity < or =125 ms), there was no significant effect of phase relationship on PSSAs, but still an approximately constant shift with respect to the reference value. Variation of the repetition rate resulted in almost constant shifts in PSSA of approximately 3 degrees between 1 and 4 Hz and a linear decrease (slope 0.27 degrees /Hz) with higher repetition rates. These results suggest a spatio-temporal 'window' for auditory-visual integration, that extends over approximately 100 ms and approximately 3 degrees : when auditory and visual stimuli are within this window, they are always perceived as spatially coincident. These psychophysical findings may be related to properties of bimodal neurons such as have been demonstrated by neurophysiological recordings in midbrain and cortex.