Objective fidelity evaluation in multisensory virtual environments: auditory cue fidelity in flight simulation

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Perception Psychology

Background:

• Traditional virtual environment fidelity evaluation often relies on physical accuracy.
• Human performance in simulated environments is crucial for effective training and operation.
• Multisensory perception research offers frameworks for objective human performance measurement.

Purpose of the Study:

• To propose a human-performance-centered and task-specific framework for objective fidelity evaluation of virtual environments.
• To demonstrate the framework's application in a flight simulation environment.
• To quantify the contribution of individual sensory cues to human performance in different tasks.

Main Methods:

• Utilized principled experimental paradigms and behavioral models from multisensory perception research.
• Conducted three case study experiments in a flight simulation environment: cue detection/categorization, target tracking, and transferrable learning.
• Measured human performance and evaluated the contribution of auditory and kinematic motion cues.

Main Results:

• The contribution of individual sensory cues to human performance is highly task-dependent.
• Auditory cues, though discriminable and optimally integrated in one task, did not aid target tracking without training.
• Auditory cues significantly improved performance with training in a transferrable learning task.

Conclusions:

• Objective fidelity evaluation of virtual environments necessitates a task-specific analysis of cue contributions.
• Human performance metrics are essential for understanding the true fidelity of simulations.
• The framework provides a robust method for evaluating sensory cue impact on performance in virtual environments.