Guiding Auditory AR: A VR-Based Evaluation of Visual and Tactile Cues for Sound Localization for Cochlear Implant...

The inability of cochlear implant (CI) users to accurately localize sound sources poses daily safety risks and impedes social communication. While future augmented reality (AR) glasses are an ideal platform for delivering real-time visual and tactile assistive cues, their potential is hampered by a critical knowledge gap: how to design effective, non-distracting interfaces for this specific user group. To address this, we introduce a novel methodological approach using a virtual reality (VR) platform with high-fidelity color passthrough. This platform functions as an ecologically valid yet fully controllable testbed that faithfully simulates a future AR experience. Within this environment, we conducted a within-subjects study with 16 CI users to systematically evaluate three non-auditory cueing strategies: 1) Visual-Only, using color-coded peripheral widgets; 2) Tactile-Only, via a custom 8-channel vibrotactile array; and 3) a combined visuo-tactile Multimodal approach. We rigorously measured localization accuracy (absolute angular error), reaction time, front-back confusion rates, and subjective cognitive load (NASA-TLX). Our results reveal a critical performance-load trade-off: the multimodal strategy yielded the highest accuracy, reducing the mean error from 85.3° to 8.1°, but also imposed the greatest cognitive load. Conversely, the tactile-only strategy offered a compelling balance, significantly improving performance while being rated as the least cognitively demanding. From this analysis, we derive foundational insights and a set of evidence-based design guidelines to inform the creation of more effective and usable auditory awareness systems on future AR platforms.