Virtual Reality Mediated Haptic Robot Telemanipulation Without Cameras: Technology and Pilot Study

Robot telemanipulation is widely used to maintain effective robotic control by keeping a human operator in the loop. Traditional camera-based systems have achieved remarkable success, but they can fail under challenging conditions such as poor lighting, smoke, haze, or occlusions. This work introduces an alternative approach and demonstrates the feasibility of precise pick-and-place telemanipulation in the most extreme scenario: operating without any cameras or any other real-world visual input. Our system relies solely on tactile-driven 3D object reconstruction streamed to a Virtual Reality (VR) headset, combined with haptic feedback delivered through a hand exoskeleton. In our experiments, an expert operator wearing a VR headset successfully performed pick-and-place tasks with an 80% success rate across 90 trials involving three object configurations. To our knowledge, this is the first demonstration of real-world camera-free (blind) dexterous telemanipulation. While not intended as a universal replacement for vision-based systems, the presented approach enables effective telemanipulation in environments where visual sensing is unreliable or unavailable, thereby expanding the operational domain of robotic telemanipulation systems.