Multimodal and Multidimensional Tactile Perception for Universal Robotic Sophisticated Tasks

Human hands have rich tactile senses with multimodal and multidimensional perceptions for hand dexterous manipulation. However, current robotic tactile sensory modality and dimensionality are deficient and hardly meet the harsh demands for diverse manipulation tasks. Here, we report a multimodal and multidimensional tactile sensor based on unified thermo-sensation, enabling comprehensive perceptions of contact, matter type, slip/slide, texture/scratch, temperature, and multidimensional force to meet the general-purpose perceptual requirement across diverse sophisticated robotic tasks. The sensor is compact, flexible, ultralight (0.33 g), and integrated on the robot's fingertip. Ultrafast contact and slip detections (~5 ms) of the sensor enable instantaneous action triggering of robots to ensure agile manipulations, while its precise texture/scratch detection (at the ~20 μm scale) surpasses human touch sensitivity. The sensor also provides decoupled and high-precision measurements of 3-axis forces with fast response, low hysteresis, and low crosstalk, ensuring dexterous action control of robots. Furthermore, we propose a framework of tactile perception-control synergy, endowing robots with self-adaptive action-on-feeling like humans. Leveraging the multimodal and multidimensional tactile perception as well as the synergy with adaptive control, robots autonomously accomplish various sophisticated tasks of precise scratch inspection, fine assembly, and plug-socket insertion, etc.