Human-taught sensory-control synergy for universal robotic grasping

Universal grasping is essential but challenging in robotic manipulations, particularly for humanoid robots with multifingered hands. To learn skills of dexterous manipulations from humans, we propose a sensory-control synergy (SCS) approach mimicking the human grasping experience. We develop a tactile glove worn on a human hand to capture multimodal tactile data (contact, slip and pressure) during human grasping demonstrations. Emulating human neurocognition, the multimodal tactile data are encoded into semantically explicit grasping states by neural-network computing. Drawing on human motor control strategies, an experience-based fuzzy controller is built to swiftly convert semantic grasping states into grasping actions. Benefiting from the semantization of grasping states, the SCS model is highly logicalized and generalizable, can be data-efficiently built by non-experts and readily transferred to robots for accomplishing universal robotic manipulation. The robot with SCS achieves an average success rate of 95.2% in grasping diverse objects of daily life, including slippery, fragile, soft and heavy objects. In dynamic disturbance and complex tasks, the robot autonomously manipulates using its adaptive SCS, demonstrating human-like universal grasping capabilities.