Related Experiment Video
Updated: Sep 2, 2026

Photorealistic Learned Landscapes for Augmented Reality
Published on: June 27, 2025
Depth-based visual capture of sound in virtual reality is driven by perceived distance
Maggie K McCracken1, Macy E Rowland2, Hunter C Finney2
1Department of Psychology, University of Utah, 380 S 1530 E, Salt Lake City, UT, 84112, USA. maggie.mccracken@psych.utah.edu.
Abstract:
Visual information typically dominates auditory cues in spatial localization, particularly in depth, leading to the proximity image effect (PIE) in which displaced sounds are perceived closer to plausible visual objects than they actually are (visual capture). It is unclear whether fundamental properties of audiovisual integration generalize to virtual environments where spatial perception is systematically distorted. Across two experiments, we tested whether real-world sound sources are visually captured by virtual objects in immersive virtual reality (VR). Participants completed a perceptual-matching task in which they indicated when a real-world sound source aligned with the depth of a virtual object. In Experiment 1, visual capture reliably occurred; however, the effect was opposite to real-world findings, with stronger capture happening when the sound originated in front of the visual object rather than behind it. Experiment 2 replicated this pattern and revealed significant visual distance underestimation measured with a blind-walking task. When judgments of auditory-visual co-location were interpreted relative to the participants' perceived visual distance rather than the actual physical distance, the expected real-world symmetry reemerged. These results demonstrate that core principles of multisensory integration are preserved in VR, but VR-specific biases, such as the visual underestimation of distance, may alter the direction of the established effects.
Related Concept Videos
Depth Perception and Spatial Vision
Echo
Imagine the sound is reflected back to the ears. Assuming that the source is very close to the human, the difference between hearing the two sounds—the emitted sound and the reflected sound—may be more than the minimum time for perceiving distinct sounds. If this is the case, then the...
Perceiving Loudness, Pitch, and Location
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
The Cochlea
Sound Waves
Sound waves are longitudinal in most fluids because fluids cannot sustain any lateral pressure. In solids, however, shear forces help in propagating the disturbance in the lateral direction as well. Hence,...
Auditory Pathway
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
