Related Experiment Video
Updated: Jun 10, 2026
09:46
MPI CyberMotion Simulator: Implementation of a Novel Motion Simulator to Investigate Multisensory Path Integration in Three Dimensions
Published on: May 10, 2012
Multisensory integration with a head-mounted display: background visual motion and sound motion.
William J Harrison1, Matthew B Thompson, Penelope M Sanderson
1The University of Queensland, St Lucia, Queensland, Australia.
Human Factors
|July 27, 2010
Summary
Mobile head-mounted display (HMD) users struggle with integrating visual and auditory information when experiencing background visual motion and relative sound movement. This impacts performance in healthcare and military applications.
Area of Science:
- Human-Computer Interaction
- Multisensory Integration
- Virtual Reality
Background:
- Mobile head-mounted display (HMD) users often face challenges integrating auditory and visual information, particularly when sound originates from a fixed speaker.
- Previous studies indicate poor task performance in mobile HMD users, but the underlying reasons remain unclear.
Purpose of the Study:
- To investigate the impact of background visual motion and relative sound movement on HMD user performance in audiovisual integration tasks.
- To identify factors contributing to performance degradation in mobile HMD users.
Main Methods:
- Participants performed an audiovisual event counting task using an HMD with sounds from a free-field speaker.
- Task completion was assessed under conditions of walking, sitting, and within a mobile room allowing independent manipulation of visual and auditory motion.
Main Results:
- Accuracy in counting audiovisual events significantly decreased when participants were walking compared to sitting (p = .032).
- A trend towards decreased accuracy was observed when participants experienced combined background visual motion and relative sound movement compared to sitting (p = .058).
Conclusions:
- Multisensory integration is significantly impaired in HMD users experiencing simultaneous background visual motion and relative sound movement.
- These findings suggest that eye reflexes may be a critical factor in reduced performance.
- The results are relevant for mobile HMD applications in healthcare and military settings.
Related Concept Videos
01:15Depth Perception and Spatial Vision
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
01:29The Vestibular System
The vestibular system is a set of inner ear structures that provide a sense of balance and spatial orientation. This system is comprised of structures within the labyrinth of the inner ear, including the cochlea and two otolith organs—the utricle and saccule. The labyrinth also contains three semicircular canals—superior, posterior, and horizontal—that are oriented on different planes.