From Sight to Touch: Haptic Sensory Integration Can Facilitate Multi-Limb Coordination

Humans possess an innate ability to seamlessly coordinate movement across multiple limbs, whether driving a motor vehicle, playing a musical instrument, or performing other daily tasks. Here, supplemental sensory information, such as haptic feedback, can enhance this coordination in applications ranging from controlling teleoperated robots to prosthetic limbs and collaborative robotics. Yet, a critical gap remains in our understanding of how visual and haptic information are integrated within sensorimotor feedback systems, as well as the extent to which these sensory channels may serve as substitutes for one another. To address this gap, we conducted an experiment investigating how sensory feedback can be incorporated in a multi-limb coordination task. To determine the degree to which visual or haptic feedback dominates in multi-limb coordination, 25 participants performed a virtual cursor-to-target task using both upper limbs (via a joystick controller) and one lower limb (via a foot pedal controller). Throughout the task, we systematically manipulated visual and haptic feedback, using a vibrotactile haptic feedback algorithm that delivered task-relevant information to all three limbs. We assessed participants' task performance measures relating to trial success rates, completion times, ability to move their limbs in coordination, and overall movement efficiency. Additionally, participants completed a cognitive workload questionnaire to evaluate their perceived task difficulty level and cognitive demands. Our findings indicate that haptic feedback can effectively substitute for one degree of visual information (cursor movement along one axis). We found no significant difference between conditions where all visual cues were presented in the task and the condition where one aspect of visual feedback was replaced by haptic feedback. These results suggest that haptic feedback can, to an extent, serve as a viable alternative to visual feedback in multi-limb coordination tasks.