Related Experiment Video
Updated: Jun 26, 2026
09:46
MPI CyberMotion Simulator: Implementation of a Novel Motion Simulator to Investigate Multisensory Path Integration in Three Dimensions
Published on: May 10, 2012
Multisensory integration with a head-mounted display: sound delivery and self-motion
Matthew B Thompson1, Penelope M Sanderson
1ARC Key Centre for Human Factors, The University of Queensland, St Lucia, Queensland, Australia.
Human Factors
|December 30, 2008
Summary
Sound delivery method impacts multisensory integration while walking with a head-mounted display (HMD). Free-field sound significantly degrades performance when walking, unlike private earpiece audio, suggesting motion and spatial conflicts.
Area of Science:
- Human-Computer Interaction
- Auditory Perception
- Multisensory Integration
Background:
- Head-mounted displays (HMDs) are increasingly used for mobile work, with potential benefits from auditory support.
- The optimal method for delivering sound (free-field vs. earpiece) and the impact of user motion remain unclear for HMD applications.
Purpose of the Study:
- To investigate how sound delivery method affects multisensory integration in users walking with a head-mounted display (HMD).
- To determine if auditory information is better integrated via earpiece or free-field sound during locomotion with an HMD.
Main Methods:
- Participants performed a sound-visual mismatch detection task while using an HMD.
- Sound stimuli were delivered either through an earpiece or in the free field.
- Performance was compared between seated and walking conditions for both sound delivery methods.
Main Results:
- Participants performed equally well on the mismatch task whether sitting or walking when sound was delivered via earpiece.
- Performance significantly worsened when participants walked compared to sitting while using free-field sound (p = .006).
Conclusions:
- Worse performance with free-field sound during walking may stem from spatial and motion discrepancies between head-referenced visuals and world-referenced audio.
- Self-motion effects on multisensory integration tasks with HMDs warrant further research.
- Free-field sound delivery in HMD applications may lead to performance degradation due to changing sound source localization during user movement.
Related Concept Videos
01:29The Vestibular System
The vestibular system is a set of inner ear structures that provide a sense of balance and spatial orientation. This system is comprised of structures within the labyrinth of the inner ear, including the cochlea and two otolith organs—the utricle and saccule. The labyrinth also contains three semicircular canals—superior, posterior, and horizontal—that are oriented on different planes.
01:15Depth Perception and Spatial Vision
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.