Sensory Substitution and Embodiment Shape Performance and Immersion in Teleoperated Settings.

Area of Science:

• Human-Computer Interaction
• Robotics
• Neuroscience

Background:

• Telemanipulation involves remote control with limited sensory input, often lacking haptic feedback.
• Embodiment and Presence are crucial for effective teleoperation, allowing tools to feel like extensions of the user.
• Sensory Substitution (SS) can overcome sensory limitations by translating information across modalities.

Purpose of the Study:

• To investigate the learning of Sensory Substitution (SS) in telemanipulation tasks.
• To examine the influence of SS on Embodiment and Presence.
• To quantify the impact of SS on performance metrics and workload.

Main Methods:

• A Visual-Tactile Sensory Substitution Task was developed with 40 participants.
• A within-subjects design compared force feedback (Control) with no force feedback (SS Condition).
• Performance (errors, time, force) and physiological (pupil dilation, blinks) measures were collected, alongside subjective assessments.

Main Results:

• The SS Condition showed reduced task time and increased errors but comparable maximal applied force.
• Blink rate modulated with task learning, while pupil dilation remained stable.
• Structural Equation Modeling confirmed that perceived SS positively impacted Embodiment, Presence, and Learning.

Conclusions:

• Sensory Substitution (SS) effectively supports telemanipulation by enhancing Embodiment and reducing task completion time.
• SS modulates sensorimotor learning strategies, crucial for efficient remote operations.
• This approach offers a viable method to mitigate sensory limitations in telepresence tasks.