Sensory augmentation for a rapid motor task in a multisensory environment

Area of Science:

• Neuroscience
• Human-Computer Interaction
• Robotics

Background:

• Sensory substitution and augmentation systems (SSASy) offer alternative sensory input pathways.
• Previous SSASy research focused on untimed, single-sense tasks.

Purpose of the Study:

• Investigate SSASy utility in rapid, dynamic motor actions within a multisensory context.
• Assess SSASy integration with existing sensorimotor capabilities.

Main Methods:

• Participants engaged in a virtual reality air hockey game using motion controls.
• Training involved a novel audio cue (SSASy) for puck localization.
• Performance was evaluated using SSASy, degraded vision, or combined sensory inputs.

Main Results:

• Participants demonstrated significantly improved target striking accuracy when coordinating vision with the SSASy.
• Combined sensory input (vision + SSASy) outperformed single-cue performance (t(13)=9.16, p<.001, Cohen's d=2.448).

Conclusions:

• Individuals can flexibly adapt SSASy for time-critical, high-precision motor tasks.
• SSASy can augment, not just replace, existing sensorimotor skills.
• Potential applications include aiding individuals with vision impairment and enhancing general human perceptual-motor performance.