Overdriving Visual Depth Perception via Sound Modulation in VR

Area of Science:

• Human-computer interaction
• Neuroscience
• Virtual Reality

Background:

• Depth perception relies on integrating visual and auditory cues.
• Vergence movements control eye fixation for depth perception in stereoscopic 3D vision.
• Auditory cues like loudness and interaural differences offer complementary depth information.

Purpose of the Study:

• To investigate the interaction between visual and auditory cues in virtual reality.
• To examine how contradictory auditory information can influence visual depth perception.
• To assess audio-driven depth enhancement for improved virtual reality experiences.

Main Methods:

• Introducing spatial discrepancies between visual targets and auditory cues in VR.
• Presenting conflicting visual and auditory depth information to participants.
• Conducting psychophysical studies to evaluate depth overdriving efficiency and gaze retargeting speed.

Main Results:

• Audiovisual fusion biases depth perception toward the intended location.
• Audio-driven depth enhancement counteracts depth compression, reducing vergence magnitude.
• Significant speedup in gaze retargeting was observed with successful audio manipulation.

Conclusions:

• Contradictory auditory cues can effectively "overdrive" visual depth perception in VR.
• This audiovisual integration improves depth perception and facilitates faster visual search.
• The findings suggest potential applications in mitigating the vergence-accommodation conflict (VAC) in VR.